SYSTEMATIC REVIEW article

Front. Psychiatry, 07 June 2021

Sec. Addictive Disorders

Volume 12 - 2021 | https://doi.org/10.3389/fpsyt.2021.675237

Problematic Gaming and Sleep: A Systematic Review and Meta-Analysis

  • 1. Department of Psychosocial Science, University of Bergen, Bergen, Norway

  • 2. Norwegian Competence Centre for Gambling and Gaming Research, University of Bergen, Bergen, Norway

  • 3. Optentia, Vaal Triangle Campus of the North-West University, Vanderbijlpark, South Africa

  • 4. Norwegian Competence Center for Sleep Disorders, Haukeland University Hospital, Bergen, Norway

  • 5. College of Education, Psychology, & Social Work, Flinders University, Adelaide, SA, Australia

Article metrics

View details

87

Citations

32,9k

Views

5,3k

Downloads

Abstract

Problematic gaming has been linked to poor sleep outcomes; however, these associations have not yet been synthesized quantitatively. This review employed a meta-analysis to investigate the relationship between problematic gaming and sleep-related outcomes. A search of Medline, Embase, Web of Science, PsycINFO, and Google Scholar identified a total of 763 studies, including 34 studies (n = 51,901 participants) eligible for inclusion. Papers were included if available in any European language, addressed problematic gaming, contained original data, and provided sufficient data for calculation of effect sizes. Two researchers independently extracted data using pre-defined fields including quality assessment. Sleep-related outcomes were meta-analyzed for sleep parameters that were reported by 5 or more papers. Significant overall effects were found for sleep duration (g = −0.238, 95% CI = −0.364, −0.112), poor sleep quality (OR = 2.02, 95% CI = 1.47, 2.78), daytime sleepiness (OR = 1.57, 95% CI = 1.00, 2.46) and sleep problems (OR = 2.60, 95% CI = 1.94, 3.47). Between-study heterogeneity was detected for all meta-analyses. Subgroup analyses showed a higher inverse effect size for adolescent samples compared to adult or non-specific age samples in terms of sleep duration. For daytime sleepiness, a larger effect size was found for studies based on single-item sleep measures compared to multi-item sleep measures. For sleep problems, the subgroup analysis showed the opposite with a higher effect size for studies based on single-item sleep measures than multi-item sleep measures. Across all sleep parameters, problematic gamers consistently reported a more adverse sleep status than non-problematic gamers.

Systematic Review Registration:https://www.crd.york.ac.uk/PROSPERO/; record ID: CRD42020158955.

Introduction

Gaming is a popular leisure activity worldwide (1). While gaming can be a beneficial activity for most individuals (2, 3), gaming becomes problematic for a minority of players who experience negative consequences of excessive gaming (4, 5). Such negative outcomes include depression, anxiety, loneliness, somatization, reduced quality of life, and poor academic achievement (510). Problematic gaming has largely been viewed as a behavioral addiction and classified based on the presence of addiction symptoms (i.e., salience, mood modification, tolerance, withdrawal, conflict, and relapse) (11). The inclusion of Internet Gaming Disorder (IGD) in the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) as a condition for further study (12) resulted in increased research on the condition. Further, problematic gaming received formal recognition as a mental/behavioral disorder by the World Health Organization (WHO) with the inclusion of Gaming Disorder (GD) in the 11th edition of the International Classification of Diseases [ICD-11; (13)]. Given the widespread definitions, lack of standardized measuring tools, and cutoff values to indicate problematic/pathological behavior, the prevalence rates of problematic gaming have varied significantly across studies ranging from 0.2 to 34% (14). Recent meta-analyses have estimated the average prevalence rate of problematic gaming to be 3.1–4.6% (15, 16). The empirical data have been more consistent showing that boys and men report more problems related to video gaming compared to girls and women and that problematic gaming is more common among younger than older subjects (10, 14, 17). It should be noted that the field of problematic gaming, including the diagnoses in the DSM-5 and ICD-11, involves several controversies, where both the conceptualizations and the measurements of problematic gaming have been criticized (1820). For instance, some researchers have asserted that the etiology of problematic gaming is unknown and that the state of problematic gaming may reflect underlying difficulties (e.g., depression) rather than a problem in its own right (18). The controversies regarding problematic gaming substantiate the need for a better understanding of the relationship between problematic gaming and other difficulties.

As the levels of gaming have increased over the past years so have the prevalence rates of sleep problems (21). Furthermore, an increasing number of studies have found associations between problematic gaming and sleep problems (2224). However, the strength of the relationship between problematic gaming and sleep problems has been inconsistent across studies and the directionality/causality remains unclear. Given the importance of sufficient sleep duration and quality for optimal functioning, subjective well-being, and good health, identifying potential determinants of sleep problems (e.g., problematic gaming) are of important scientific and practical interest. Moreover, as the negative outcomes associated with problematic gaming are similar to the negative outcomes observed in individuals with sleep problems (e.g., depression, reduced quality of life, poor academic achievement), insight into the sleep characteristics of problematic gamers could contribute to a better understanding of the phenomenon of problematic gaming.

There are a number of possible mechanisms by which gaming may influence sleep (25). As proposed by the media displacement hypothesis (26, 27), gaming could displace sleep directly as the individual chooses to engage in games over sleep, or indirectly by disregarding behaviors that are essential for good sleep hygiene (e.g., physical activity). For problematic gamers, the displacement may not be by choice but rather driven by an inability to stop playing. Arousal is another possible mechanism both due to social engagement, structural characteristics of the games (e.g., high event frequency), and the thrill related to winning or losing, which may interfere with sleep. The artificial blue-spectrum light emitted by screens projecting the visual gaming content may also directly enhance alertness and arousal (28), and suppresses nighttime melatonin secretion, important in regulating the sleep-wake cycle (29, 30). Followingly, late-night gaming may delay the sleep phase, making it difficult to fall asleep at needed or wanted times. Prolonged gaming may also negatively affect sleep by creating physical discomforts such as muscular pain and headache (31). Lastly, sleep may also be interfered by exposure to electromagnetic fields that are emitted by wireless gaming devices, which may alter the total sleep time, sleep efficiency, sleep architecture, as well as inhibit the secretion of melatonin (3234).

Taken together, problematic gaming behavior may lead to shorter sleep duration, poorer sleep quality, delayed sleep phase, and problems initiating or maintaining sleep. However, the direction of the relationship could also be reversed. It is possible that individuals with insomnia and others experiencing sleep difficulties may use gaming to cope with sleeplessness and as such develop problematic gaming patterns. Also, there could be a bi-directional (reciprocal) relationship, meaning that problematic gaming could cause sleep problems which in turn could further fuel further problematic gaming and the other way around. Alternatively, it is possible that both problematic gaming and sleep problems are caused by common third variables. For example, both problematic gaming and insomnia have been shown to be strongly associated with depression (5, 35).

Given the growth in empirical evidence on the topic, there is a need for an updated review of the current knowledge on the sleep outcomes associated with problematic gaming. Moreover, there is to date no available meta-analysis quantifying the strength of the potential relationship between problematic gaming and sleep-related outcomes across studies. The following research questions were thus addressed: (1) What characterizes the studies conducted so far on the relationship between problematic gaming and sleep?, (2) what is the evidence for a causal relationship between problematic gambling and sleep? and (3) what is the strength of the association between problematic gaming and sleep problems? In an effort to consolidate the knowledge on the phenomenon the current study will use problematic gaming as an umbrella term for the wide range of related constructs used in the literature. Hence, problematic gaming is in this paper defined as a pattern of playing video games that could negatively influence physical and mental health and/or interfere with daily activities. It entails both online and offline gaming behavior and all types of digital gaming devices.

Methods

Search Strategy and Inclusion Criteria

A systematic literature review and meta-analysis were conducted in accordance with the PRISMA guidelines (36, 37). The meta-analysis was pre-registered at the PROSPERO International prospective register of systematic reviews (https://www.crd.york.ac.uk/PROSPERO/; record ID: CRD42020158955). A search strategy was developed by the authors and the literature searches were completed on January 11th, 2021, conducted using relevant keywords in Web of Science, Medline, Embase, PsycINFO, and Google Scholar electronic databases. Search words are displayed in Table 1. The search words were entered similarly in each database without any limits, filters, or use of specific MeSH-items. Additionally, the reference lists of relevant studies were examined. The search strategy yielded a total of 1,360 results, with the following result in each database: Web of Science (543 results), Medline (419 results), Embase (186 results), and PsycINFO (212 results). Due to a large number of results provided by Google Scholar (more than 18,800 hits), only the first 30 pages of results were reviewed; identifying five additional manuscripts.

Table 1

Keywords/termsOperandKeywords/terms
Problematic gamingSleep
“Problem* gaming”Sleep*
“Gaming addiction”Insomnia
“Game addiction”Circadian
“Gaming disorder*“Morningness-eveningness”
“Internet gaming”“Delayed sleep”
“Internet game”“Social jet-lag”
“Online gaming”“Wake after sleep onset”
OR“Online game*ANDORSnoring
“Video gaming”Hypersomnia
“Video game*
“Computer gaming”
“Computer game*
“Excessive computer use”
“Risk* gaming”
“Pathological gaming”
“Pathological internet”

Search items.

*

Truncation.

For inclusion in the present review, the studies had to fulfill the following eligibility criteria: (1) The full manuscript was available in a European language (e.g., English, German, Spanish, French, Scandinavian languages, etc.), (2) The study addressed problematic gaming specifically; (3) The study contained original quantitative data on the relationship between problematic gaming and at least one sleep variable, and (4) the study reported estimates of, or sufficient data to calculate an effect size of, the strength of this relationship. No further restrictions in terms of publication time or design were implemented. In instances where studies reported insufficient data, the corresponding authors were contacted to provide the required data. Studies not reporting data on gaming specifically, but rather on problematic screen or internet behaviors were excluded.

Data Extraction and Coding Procedures

The two main outcomes of interest were measures of problematic gaming and sleep-related outcomes assessed by self-report questionnaires, clinical interviews, or objective measurements. The data were extracted and coded using a coding schema encompassing the manuscript: Author(s), manuscript type, year published; study setting: Design, country, continent, year of data collection, ethical approval, conflict of interest; sample: Sample size and characteristics of participants, methodology: Collection method, instruments, instrument reliability, sleep parameter; and results: Type of estimate, effect size, and adjustment of confounding. Effect sizes were either extracted directly from the original publications or manually calculated. The extraction procedure was conducted by JHK and EKE who independently coded the studies. Discrepancies between the two reviewers were resolved by discussions. In studies categorizing subgroups of problematic involvement in games based on the endorsement of a number of symptoms (e.g., a polythetic scoring to indicate less severe “problematic gamer”/“at-risk” and a monothetic system to identify those who were “addicted”/“disordered”), the effect sizes of all/both groups were collapsed by calculating a mean effect size using a fixed effect-model ensuring that the control/contrast group were not counted more than once (38, 39). Also, one study reported data separately for sibling dyads (40). In this case, the effect sizes (and descriptive data) were collapsed for the dyads and corrected for the number of participants before entered into the meta-analysis. One study assessed problematic gaming using both the DSM-5 IGD and the ICD-11 GD criteria (41). In this instance, only the data based on the IGD criteria were included, for comparability with other studies in the present review. In cases where subscale data of the Pittsburgh Sleep Quality Index (PSQI) were reported, the global score estimate was coded as sleep quality while the subscale data were independently coded in terms of their respective sleep parameters. It should be noted that the associations between problematic gaming and sleep-related outcomes in the present meta-analysis are based on unadjusted relationships. Although some studies controlled for potential confounders, this was not conducted consistently across studies and the number and content of confounders adjusted for also varied. Thus, in order to extract comparable effect sizes for each study, we extracted unadjusted estimates and not parameters (e.g., standardized beta regression coefficients) correcting for confounders.

Quality Assessment

Two reviewers (JHK and EKE) independently assessed the quality of the included studies using the Newcastle-Ottawa Quality Assessment Scale (NOS) adapted for cross-sectional studies (42, 43). The NOS assesses study quality in three categories; (1) the sample selection process; (2) comparability between groups; and (3) the ascertainment of the results. To determine the level of inter-rater reliability in the quality assessment procedure, Cohen's Kappa statistic was calculated and was found to be 0.667 (p < 0.001), indicating substantial agreement between the two reviewers. In the cases of disagreement between reviewers, consensus was sought through discussions.

Meta-Analyses

Meta-analyses were conducted to calculate overall effect sizes for each sleep parameter separately. Based on previous reviews of the literature we expected a considerable variance in both terminology and the instruments used to measure problematic gaming as well as sleep parameters, as well as divergence in terms of study populations (44, 45). As such, the meta-analyses were a priori planned to use random-effects models for any sleep parameter that yielded five or more studies. If <5 specific studies could be identified for a given sleep parameter, the parameter was not included for meta-analysis as the results were judged to be too unstable and since random-effects models in such cases could yield an inappropriate estimation of the between-study variance. To quantify between-study variance, tests of heterogeneity were calculated using the Q and I2-statistics. The Q-statistic is a measure of the total observed study-to-study variation, and a significant Q indicates the presence of heterogeneity between the studies. The I2-statistic is a measure of percentage which quantifies the total amount of variability in a set of effect sizes that is a result of true differences between the studies. An I2 percentage of 25, 50, and 75 can roughly be interpreted as low, medium, and high levels of true heterogeneity, respectively (46).

Subgroup meta-analyses were conducted in an effort to explain the between-study variance. The impact of participant age was explored by creating a dichotomous moderator variable that separated studies of adolescents vs. adults. The weighted mean age was 15.1 and 23.6 for the two subgroups, respectively. In addition, a second dichotomous moderator variable separating studies that investigated sleep with a standardized multiple-item questionnaire and studies relying on single items (or own created) was created in order to assess the impact of this methodical discrepancy on the overall effect size. The subgroup analyses were based on mixed-effects models, using a random-effects model within subgroups (pooling within-group estimates of tau-squared) and a fixed-effect model across subgroups (38).

Publication bias was assessed for each sleep parameter by visual inspection of funnel plots and statistically by the Orwin's fail-safe N (47) and the Duval and Tweedie's trim and fill procedure (48, 49). The Orwin's fail-safe N is an estimate of how many missing studies with a specified effect are needed to bring the overall effect statistically to a pre-set level. If the N is low, there is concern that the whole overall effect is an artifact of publication bias as it is likely that some studies are missing due to publication bias. In the current study, g = 0.2 and OR = 1.2 were set as criteria for “trivial” effects, and g = 0.0 and OR = 1.0 were set as the mean of the missing studies. Lastly, Duval and Tweedie's trim and fill procedure was used to assess the magnitude of bias on the overall effect. It complements the funnel plot by imputing theoretically missing studies and adjust the overall effect size to the best estimate of an unbiased effect size. The extrapolated overall effect sizes are reported in terms of Hedges' g and odds ratios (OR). The Hedges' g is an estimation of the standardized mean difference between groups, which has the advantage over Cohen's d that it corrects for bias caused by small sample sizes. Hedges' g is interpreted in line with Cohen's (50) convention of small (0.2), medium (0.5), and large (0.8) effects. An OR of 2.0, 3.0, and 4.0 could be considered as small, moderate, and strong effects, respectively (51). All statistical analyses were conducted using Comprehensive Meta-Analysis, version 3.3.070 (Biostat, Inc., 2014).

Results

Study Selection

Figure 1 presents a PRISMA flow diagram of the study screening and selection process. After removing duplicates (n = 597), the remaining (n = 763) titles and abstracts were independently screened by two reviewers (JHK and a research assistant), resulting in 101 studies for full-text eligibility evaluation. From this evaluation, 67 records were excluded resulting in 34 studies that met the eligibility criteria and were included in the review (see Figure 1 for reasons). From this pool, a total of 57 estimates (or results) on the relationship between problematic gaming and ten different sleep parameters were identified. Twelve studies reported sleep duration, 11 reported sleep quality, seven reported daytime sleepiness, 15 reported sleep problems, one reported sleep loss, one reported morningness-eveningness, one reported delayed sleep phase disorder, two reported bedtimes and wake-up times, and four studies reported other sleep-related outcomes.

Figure 1

Descriptive Characteristics of the Included Studies

Table 2 summarizes the study characteristics and associations. Of all the 34 studies that met the eligibility criteria, 33 manuscripts were peer-reviewed research papers while one manuscript comprised an unpublished master-thesis (56). All manuscripts were published in English, except for one which was published in French (57). Regarding the study designs, one study used a longitudinal design (77), while the vast majority of studies (n = 32) were cross-sectional. Additionally, one study was a cohort study (65), but the sleep data were only collected at the first wave – making the data included in the present synthesis cross-sectional. The studies were published in the time frame of 2009–2021. The included studies yielded a summarized pool of 51,901 participants, with sample sizes ranging from 60 to 15,168 participants. The study populations came from 26 different countries with a majority originating from Europe. The participants' age ranged from 10 to 74 years with a weighted grand mean of 16.1 years. The proportion of girls/women ranged from 0 to 70.5%, with a weighted overall sex distribution of 30.7% girls/women for all the included studies. After merging subgroups of problematic gamers (e.g., “problematic”/“at-risk” and “addicted”/“disordered”), the included samples yielded a total of 5,625 individuals who were categorized as problematic gamers. Prevalence rates of problematic gaming varied widely in the samples ranging from 1.2 to 73.9% (weighted mean = 16.0%), while three studies had comparison groups of similar sample size (41, 71, 76). The majority of samples (n = 18) were recruited from primary- / high schools, while eight samples were recruited from college/universities (41, 5355, 66, 75, 76, 80), three were recruited from gaming communities (24, 52, 74), one was recruited from two pediatric lipid and obesity treatment clinics (77), one was recruited through social media (61), one was recruited in “non-working contexts” [e.g., pubs, sports associations, recreational places; (59)], and two samples were recruited using random population sampling (56, 79).

Table 2

ReferencesCountrySample size (n)Mean age (SD)/rangeSex (% females)PG prevalence % (subgroups)aPG instrument (α)PG cutoffSleep instrument (α)Sleep cutoffSleep parameter(s) and result(s)NOS
Achab et al. (52)France44826.6 (7.1)/18–5417.327.5DAS (–)≥ 3 of 7 criteriaNon-standardized questionnaire(–)Continuous
Yes/No
Yes/No
Yes/No
Sleep duration
Sleep quality
Daytimesleepiness
Sleep deprivation
↓↓↑↑6
Akçay and Akçay (53)Turkey935/892c22.76 (2.2)/–70.5GASA-SF (0.81)ContinuousPSQI (–)PSQI (–)ESS (–)Single item
Single item
Continuous
Continuous
Continuous
Continuous
Sleep quality
Sleep duration
Daytime sleepiness
Later bedtime
Later wake-up time
↓↑↑↓NS5
Al Asqah et al. (54)Saudi Arabia22821.15 (1.6)/18–253528.1 (19.3/8.8)IGDS9-SF (–)≥2/≥5 of 9 criteriaSingle itemContinuousSleep duration5
Al Gammal et al. (55)Egypt6021.9 (2.7)/18–2653.323.3 (18.3/5.0)IGDS (–)PSQI (–)ContinuousSleep quality4
Altintas et al. (24)France21724.4 (6.98)/–19.35AIE-Q (0.84)ContinuousPSQI (0.67)High sleep quality/Low Sleep quality (HCA) and continuousSleep qualityd6
Arnesen (56)Norway81627.9 (7.36)/16–4056.14.0GASA (–)≥4 of 7 itemsBIS (0.82)ContinuousInsomnia6
Bonnaire and Phan (57)France43413.15 (0.5)/–46.88.5GASA (–)≥4 of 7 itemsSingle itemYes/NoSleep problemsNS5
Brunborg et al. (58)Norway132013.6 (0.32)/14–1552.117.1 (12.9/4.2)GASA (0.85)2–3/≥4 core symptomsSingle items (0.60–0.70)≥Once a weekSleep problems
Daytime sleepiness
↑↑9
De Pasquale et al. (59)Italy56622.74 (4.8)/18–3542.765.3IGDS9-SF (0.921)/DSM-5 IGD criteria (–)Continuous/≥ 5 DSM-5 criteriaSCL-90R (–)ContinuousSleep disorders5
Fazeli et al. (60)Iran151215.51 (2.8)/13–1843.6IGDS9-SF (0.90)ContinuousISI (0.87)ContinuousInsomnia8
Fernandes et al. (61)India
Indonesia
Malaysia
Mexico
Philippines UK
18521.59 (2.6)/–65.76GASA (0.89)ContinuousSinge itemContinuousSleep quality3
Gonzalez-Valero et al. (62)Spain57711.41 (0.5)/11–1243.218.72 (17.5/1.2)CESR (0.87)≥26/≥39 test scoreSingle itemContinuousSleep duration4
Hawi et al. (23)Lebanon52416.1 (1.0)/15–1952.144.9 (35.7/9.2)IGD−20 (0.91)≥ 50 /≥ 71 test scoreSingle itemsContinuous
Sometimes or more
Sleep duration
Woke up to continue playing
↓↑7
Kim et al. (63)South Korea23016.63 (1.0)/15–18Males only51.3OGASA (0.93)≥ 38 test scoreSingle item> <6 hours of sleepSleep duration5
King et al. (64)Australia128714.9 (1.5)/12–1850.4PTU (–)ContinuousSAMQ (–)ContinuousSleep durationWD/WE
Sleep onset latencyWD/WE
Electronic media related sleep disruption
↓↑↑5
Ko et al. (41)Taiwan207/138c25.59 (3.8)/20–3821.750*Diagnostic interview based on DSM-IGD and ICD-GD criteriaIGD: ≥5 of 9 criteria
GD: endorsement of 4, 5, 6, and 9th criteria of IGD
Diagnostic interviewYes/No
Yes/No <1, 1-2, >3 a.m. <9 a.m., 9 a.m.-12 p.m., > 12 p.m.Yes/No
Less than 4 h sleep 2 or more days a week
Insomnia
Delayedsleep phase disorder
Falling asleep at a later time
Waking up at a later time
Turning night into day
Inadequate sleep time
↑↑↑↑↑↑4
Lin et al. (65)Iran4,44215.3 (1.6)/13–1849.773.9 (47.9/26.0)GAS(A) (0.89)Latent class analysis (LCA)PSQI (–)ContinuousSleep quality9
Lin et al. (40)Iran64016.25 (2.6)/13–2433.8IGDS9-SF (–)ContinuousISI (–)ContinuousInsomnia5
Liu et al. (66)China1,04060.0GASA (0.95)ContinuousPSQI (0.64)ContinuousSleep quality4
Männikkö et al. (67)Finland29318.7 (3.4)/13–24498.2GAS(A) (0.79)≥ 4 of 7 itemsSingle itemEvery week or more oftenSleep problems4
Männikkö et al. (68)Finland77317.5 (4.4)/16–1941.1IGDT-10 (–)ContinuousSingle itemContinuousSleep duration5
Nakayama et al. (69)Japan814/549c12.21 (0.4)/12-1347.26.4IGDT-10 (0.82)≥3 test scoreSingle itemsNone–Always ≤ 21:59, 22:00-22:59, 23:00-23:59, 0:00-00-59. ≥ 1:00 h ≤ 5:59, 6:00-6:59, 7:00-7:59, 8:00-8:59, ≥ 9:00 hDaytime sleepiness (during classes)Later bedtimeWD/HD
Later wake-up timeWD/HD
↑↑↑6
Nogueira et al. (70)Portugal15211.5 (–)/10–144737.5 (33.6/3.9)DSM-5 pathological gambling (–)≥4/≥5 of 9 itemsSingle item
PDSS (–)
Continuous
Continuous
Sleep duration
Daytime sleepiness
↑↑2
Peracchia et al. (71)Italia30014.76 (1.1)/–6250*AICA-S (–)4-6 h playing a day = “Hard gamers” <1 h playing a day = “Casual gamers”PSQI (–)ContinuousSleep quality
Daytime sleepiness
Sleep efficiency
NS↓↑4
Rehbein et al. (72)Germany15.168 /7761b15.3 (0.7)/15–1648.7/0 b6.7 (4.17/2.5)KFN-CSAS-II (0.92)≥ 35/≥ 42 test scoreSingle itemsContinuous
Always having problems falling asleep the week before
Sleep duration
Sleep problems
↓↑6
Rehbein et al. (73)Germany11.00314.9 (0.7)/13–1848.91.2CSAS (0.93)≥5 of 9 criteriaSingle itemContinuousSleep problems6
Satghare et al. (74)Singapore1,08523.7 (5.3)/13–4044.5IGDQ (0.73)≥ 5 of 9 criteriaISI (0.90)≥ 10 test scoreInsomnia6
Severo et al. (75)Brazil55520.3 (5.4)/–42.556.4 (18.2/38.2)IGDS9-SF (–)≥ 16/≥ 21 test scorePSQI (–)Normal/AlteredSleep quality7
Stockdale and Coyne (76)USA17420.8 (2.2)/–*1550*IGDS (–)≥5 of 9 itemsNeuro-QOL-SD-SF (–)ContinuousSleep problems7
Turel et al. (77)USA12513.02 (2.2)/10–1733OVGA (0.87)ContinuousFitbit- actigraphyContinuousSleep duration5
Vollmer et al. (78)Turkey47112.89 (1.1)/11–1639.7CGA (0.92)ContinuousCSM (–)ContinuousEveningness7
Wenzel et al. (79)Norway3405*/16–74*1.4Single engagement item≥4 h playing a daySleep problems5
Wong et al. (80)Hongkong/China30020.89 (1.5)/–59.33IGDS9-SF (0.91)ContinuousPSQI (0.83)ContinuousSleep quality6
Yu et al. (81)China1,06612.67 (–)/12–1343.513.6DSM-5 IGD criteria (0.75)ContinuousISI (0.84)ContinuousInsomnia7

Study characteristics and associations.

↓, inverse association with problematic gaming; ↑, positive association with problematic gaming; –, not reported in the manuscript;

*

, balanced comparison groups;

NS

, not significant; NOS, Newcastle-Ottawa quality assessment scale score; α, Cronbach's alpha;

a

, collapsed subgroups of problem gamers, subgroups in brackets (e.g., problem/disordered);

b

, only boys were included in the analysis;

c

, included in the analysis;

d

, all PSQI-subscales are reported;

WD/WE

, Weekdays and weekends,

WD/HD

, Weekdays and holiday.

Problematic Gaming Assessment

Most studies (n = 33) assessed problematic gaming using self-report questionnaires, while one collected the data through diagnostic interviews (41). Seventeen studies employed self-report instruments based on the DSM-5's IGD criteria, while seven studies used the Gaming Addiction Scale for Adolescents [GASA; (5658, 61, 6567)], one study adapted the criteria of pathological gambling (70), four studies adapted the DSM-IV-TR or the ICD-10 substance dependence criteria (24, 52, 71, 72), whereas four studies used instruments that were based on general addiction symptoms (62, 63, 77, 78). In addition, one study assessed problematic gaming exclusively based on time spent playing, with four hours or more playing per day defined as “excessive gaming” (79). Thirteen studies employed a continuous measure of gaming problems, while the majority (n = 21) reported comparisons of problematic gamers and non-problematic gamers; mostly identified using polythetic cutoff values (see Table 2).

Sleep Assessment

The majority of studies (n = 32) investigated sleep using self-report questionnaires, while one study objectively measured sleep duration by employing Fitbit-actigraphy to register rest/activity cycles (77). Regarding self-report measures, 17 studies employed standardized sleep measurements where the most used instrument was the PSQI (n = 8), while five studies employed assessments for insomnia (40, 56, 60, 74, 81), and two studies assessed for daytime sleepiness (53, 70). The remaining studies assessed sleep-related outcomes using single/own-created items (see Table 2). In addition, one study collected sleep data through diagnostic interviews assessing for insomnia and delayed sleep phase (41).

Quality Assessment

The quality scores of the single included studies ranged from two to nine stars, with a mean quality score of 5.5 and a standard deviation of 1.5 (see Table 3).

Table 3

SelectionComparisonOutcome
ReferencesRepresentativeness (Max:⋆)Sample size (Max:⋆)Non-respondents (Max:⋆)Ascertainment of the exposure (Max:⋆⋆)Comparable outcome groups/Controlled for confounding factors (Max:⋆⋆)Assessment of outcome (Max:⋆⋆)Statistical test (Max:⋆)
Achab et al. (52)⋆⋆
Akçay and Akçay (53)⋆⋆
Al Asqah et al. (54)⋆⋆
Al Gammal et al. (55)⋆⋆
Altintas et al. (24)⋆⋆⋆⋆
Arnesen (56)⋆⋆
Bonnaire and Phan (57)⋆⋆
Brunborg et al. (58)⋆⋆⋆⋆
De Pasquale et al. (59)⋆⋆
Fazeli et al. (60)⋆⋆⋆⋆
Fernandes et al. (61)⋆⋆
Gonzalez-Valero et al. (62)⋆⋆
Hawi et al. (23)⋆⋆⋆⋆
Kim et al. (63)⋆⋆
King et al. (64)⋆⋆
Ko et al. (41)⋆⋆
Lin et al. (65)⋆⋆⋆⋆
Lin et al. (40)⋆⋆
Liu et al. (66)⋆⋆
Männikkö et al. (67)⋆⋆
Männikkö et al. (68)⋆⋆
Nakayama et al. (69)⋆⋆
Nogueira et al. (70)
Peracchia et al. (71)⋆⋆
Rehbein et al. (72)⋆⋆
Rehbein et al. (73)⋆⋆
Satghare et al. (74)⋆⋆
Severo et al. (75)⋆⋆⋆⋆
Stockdale and Coyne (76)⋆⋆⋆⋆
Turel et al. (77)⋆⋆⋆⋆
Vollmer et al. (78)⋆⋆⋆⋆
Wenzel et al. (79)
Wong et al. (80)⋆⋆⋆⋆
Yu et al. (81)⋆⋆

Results from the Newcastle-Ottawa quality assessment.

Meta-Analyses

Of all 34 studies and sleep parameters reviewed, sleep duration, sleep quality, daytime sleepiness, and sleep problems comprised the parameters that reached the threshold of five different effect sizes. A total of 33 studies were included in the quantitative meta-analysis.

Sleep Duration

The forest plot regarding sleep duration is displayed in Figure 2. The results from the random-effects model showed a significant overall inverse association between problematic gaming and sleep duration (g = −0.238, 95% CI [−0.364, −0.112]), suggesting that problematic gamers report shorter sleep duration than non-problematic gamers. An additional random-effects model was performed for the studies providing data on sleep duration in terms of hours and minutes (k = 6). This amounted to a difference between conditions of −20.79 min (95%, CI = [−27.31, −14.30, min.]). The Q-statistic was significant (Q = 65.7, df = 11, p < 0.001), indicating heterogeneity between the studies. The I2−statistic showed a high percentage of true between-study variance (I2 = 83.3%). A subgroup analysis comparing studies targeting adolescent populations (k = 8, weighted mean age = 15.2) and studies targeting adult or non-specific age populations (k = 4, weighted mean age = 23.7), showed that the former subgroup had a higher overall effect size (g = −0.323, 95% CI [−0.450, −0.195]) compared to the latter subgroup (g = −0.037, 95% CI [−0.222, 0.148]). The difference between the subgroups was significant (Qbet = 6.23, df = 1, p = 0.013). There was significant heterogeneity within the adolescent (Q = 17.86, df = 7, p < 0.05, I2 = 60.8%) and adult or non-specific subgroup (Q = 21.24, df = 3, p < 0.001, I2 = 85.9%). Concerning publication bias, the funnel plot was somewhat asymmetric. The trim and fill procedure suggested that two studies were missing and when imputing these the overall effect size was adjusted to g = −0.196, 95% CI [−0.317, −0.074].

Figure 2

Sleep Quality

Figure 3 displays the forest plot for sleep quality. The random-effects model showed that problematic gamers had increased odds of reporting poorer sleep quality compared to non-problematic gamers (OR = 2.02, 95% CI [1.47, 2.78]). The Q-statistic was significant (Q = 149.06, df = 10, p < 0.001), and the I2−statistic showed a high percentage of true between-study variance (I2 = 93.3%). The subgroup analysis comparing studies targeting adolescent populations (k = 3, weighted mean age = 15.2) and studies targeting adult/non-specific age populations (k = 8, weighted mean age = 22.9) did not yield any significant differences between subgroups (Qbet = 0.05, df = 1, p = 0.819). A second subgroup analysis comparing studies that used a single-item sleep assessment (k = 2) and studies using multi-item sleep questionnaires (k = 9) was conducted. The difference between the subgroups was not significant (Qbet = 2.56, df = 1, p = 0.109). The funnel plot was slightly asymmetric, and the trim and fill procedure suggested that one study was missing, adjusting the overall effect to OR = 2.09, 95% CI (1.52, 2.86). The Orwin's fail-safe N showed that 52 missing studies with zero effect are needed to reduce the overall effect to the trivial effect criterion (OR = 1.2).

Figure 3

Daytime Sleepiness

Figure 4 displays the forest plot for daytime sleepiness. The results from the random-effects model showed a significant overall effect size of OR = 1.57, 95% CI (1.00, 2.47), indicating that problematic gamers had increased odds of reporting daytime sleepiness compared to non-problematic gamers. The Q-statistic was significant (Q = 61.75, df = 6, p < 0.001), and the I2−statistic showed a high percentage of true between-study variance (I2 = 90.3%). The subgroup analysis comparing studies targeting adolescent populations (k = 4, weighted mean age = 13.3) and studies targeting adult/non-specific age populations (k = 3, weighted mean age = 24.1) did not yield any significant differences between subgroups (Qbet = 0.09, df = 1, p = 0.753). The subgroup analysis comparing studies that used a single-item sleep assessment (k = 3) and studies using multi-item sleep questionnaires (k = 4) showed that the former subgroup had a significant higher overall effect size (OR = 2.74, 95% CI [1.84, 4.08]) compared to the latter subgroup (OR = 1.01, 95% CI [0.70, 1.44]), with a Qbet = 13.30, df = 1, p < 0.001. There was still significant heterogeneity within the multi-item sleep questionnaire group (Q = 15.12, df = 3, p < 0.01, I2 = 80.1%) while there was no significant heterogeneity within the single-item sleep assessment group (Q = 1.42, df = 2, p = 0.491, I2 = 0.0%). The funnel plot was symmetric, and the trim and fill suggested that there were no missing studies; resulting in no adjustment of the overall effect size. Orwin's fail-safe N showed that 12 missing studies with zero effect are needed to reduce the overall effect to the trivial effect criterion (OR = 1.2).

Figure 4

Sleep Problems

Figure 5 displays the forest plot for sleep problems. The results from the random-effects model showed a significant overall effect size of OR = 2.60, 95% CI (1.94, 3.47), indicating that problematic gamers had increased odds of reporting sleep problems compared to non-problematic gamers. The Q-statistic was significant (Q = 121.62, df = 14, p < 0.001), indicating significant heterogeneity between the studies. The I2−statistic showed a high percentage of true between-study variance (I2 = 88.5%). The subgroup analysis comparing studies targeting adolescent populations (k = 8, weighted mean age = 15.0) and studies targeting adult/non-specific age populations (k = 7, weighted mean age = 24.6) did not yield any significant differences between subgroups (Qbet = 0.61, df = 1, p = 0.434). There was significant heterogeneity within the adolescent subgroup (Q = 115.45, df = 7, p < 0.001, I2 = 93.9%) while there was no significant heterogeneity within the adult/non-specific group (Q = 6.13, df = 6, p = 0.409, I2 = 2.1%). Finally, the subgroup analysis comparing studies that used a single-item sleep measure (k = 6) and studies using multi-item sleep questionnaires (k = 9) showed a significant difference between groups (Qbet = 7.20, df = 1, p < 0.01), where the multi-item sleep questionnaire subgroup yielded a higher overall effect size (OR = 3.47, 95% CI [2.51, 4.81]) compared to the single-item sleep measure subgroup (OR = 1.77, 95% CI [1.23, 2.65]). There was significant heterogeneity within both the multi-item sleep questionnaire group (Q = 62.92, df = 8, p < 0.001, I2 = 87.3%) and the single-item sleep assessment group (Q = 13.05, df = 5, p = 0.02, I2 = 61.7%). Concerning publication bias, the funnel plot showed a somewhat asymmetrical distribution. The trim and fill suggested that one study was missing, resulting in an adjustment to OR = 2.81, 95% CI (2.11, 3.76). The Orwin's fail-safe N showed that 76 studies with zero effect are needed to bring the overall effect down to the trivial effect criterion (OR = 1.2).

Figure 5

Discussion

The aim of this meta-analytic review was to determine the associations between problematic gaming and sleep. A total of 33 studies, including 51,430 participants across 26 countries were synthesized quantitatively. Overall, the results indicated that problematic gaming was significantly associated with shorter sleep duration (g = −0.238/raw mean difference = −20.8 min) and increased likelihood of reporting poorer sleep quality (OR = 2.02), daytime sleepiness (OR = 1.57) and sleep problems (OR = 2.60). Also, the systematic review suggests that problematic gaming is positively associated with delayed sleep phase disorder, eveningness chronotype, sleep deprivation, later bed- and wake-up times, and nocturnal awakenings. Most studies included in the review and meta-analyses were cross-sectional which precludes conclusions regarding directionality/causality.

Main Findings

All of the meta-analyzed sleep outcomes yielded significant overall effects. Following the classifications of Cohen (50) and Ferguson (51), the overall effects of g = −0.238 and ORs ranging from 1.57 to 2.60 could all be considered small. The magnitude of ORs is, however, difficult to interpret as the interpretation is dependent on the prevalence rates of the outcome (82, 83). Small effect sizes could still be relevant if the outcome variable is important (theoretically or practically) and when the outcome has high prevalence rates (84, 85). The outcome in this case (i.e., sleep problems) could be argued to be of importance due to the range of adverse consequences it involves and its high and increasing prevalence rates (21). Identifying possible determinants of sleep problems (i.e., problematic gaming) is as such of great societal importance, even if such determinants are rather weakly associated with sleep problems. Further, for the finding regarding sleep duration, one could argue that an average nightly sleep reduction of 20.8 min is clinically meaningful, as having a chronic sleep reduction of that size may have adverse consequences and is likely to be experienced as a problem by the individual (86).

The current findings are comparable to the findings in three recent meta-analyses which found significant associations between internet addiction and sleep duration (Cohens d = −0.24) and sleep problems (OR = 2.20) (87), problematic smartphone use and poor sleep quality (OR = 2.60) (88), and excessive general technology use and shorter sleep duration (Cohen's d = −0.25) (89). The latter study also found a positive significant relationship between excessive general technology use and sleep problems. This relationship yielded, however, a smaller effect size (OR = 1.33) compared to the effect size between problematic gaming and sleep problems observed in the current study (OR = 2.60). As Mei et al. (89) included a variety of different media type use in their synthesis (including general PC use, cell phone, MP3 player, tablet, and TV), this discrepancy in effect sizes might be explained by video games' inherent interactive features which may make them more arousing compared to more “passive” media types (e.g., watching TV) (9092).

Subgroup Analyses

There was significant heterogeneity in all meta-analyses. Subgroup analyses yielded no significant group differences regarding sleep quality. The lack of significant difference may, however, be a result of low power in the analyses due to the low number of studies included in each subgroup. A significant difference was found in the overall effect between studies targeting adult and adolescent populations on sleep duration where problematic gaming among adolescents was associated with shorter sleep duration than problematic gaming among adults. Given the different mean ages in the subgroups (weighted mean = 23.7 and 15.2 years) this finding could reflect that younger age is associated with shorter sleep duration among problematic gamers. This might be explained by the literature suggesting that adolescents are more prone to have a delayed sleep schedule compared to adults in general (9395). A combination of a predisposition for delayed sleep phase together with untimely late-night gaming, which may further delay the sleep schedule, might result in a more severe sleep deficit for adolescents compared to adults (96). Alternatively, as the adult samples in the current subgroup analysis comprised college/university students and young adults, this finding might also be related to the fact that this group typically have more morning flexibility (e.g., freedom in choosing which classes to attend and not) compared to primary/high school students which typically have a more predetermined and mandatory schedule and thus fewer opportunities to compensate for delayed bedtimes with later wake-up times (97).

The second subgroup analysis found that for daytime sleepiness and sleep problems, studies using standardized multi-item sleep questionnaires (e.g., PSQI/ESS) yielded statistically different effect sizes compared to studies using a single-item/own-made sleep measure. These variations in effect sizes may be attributed to the fact that standardized multiple-item measurements have more stable and robust psychometric properties (i.e., have less measuring error) compared to single-item measurements (98). This has been demonstrated empirically in a study that found that a single-item sleep disturbance measurement yielded relatively low sensitivity (i.e., ability to correctly identify sleep problems) compared to a multi-item sleep disturbance scale; suggesting that single-item sleep measurements are less able to correctly discriminate true from false positives (99). Followingly, the effect sizes derived from studies implementing single-item/own-made sleep measurements may be less valid. Thus, the overall effects on daytime sleepiness and sleep problems should be interpreted with this methodological issue in mind.

Publication Bias

The results show that all meta-analyses, except daytime sleepiness, seemed to be somewhat influenced by publication bias according to the funnel plots and trim and fill procedure. This suggests that, for these syntheses, smaller studies with larger effect sizes may be overrepresented (38, 100). The largest bias was found for sleep duration, hence the finding regarding this sleep parameter should be interpreted with some caution. For sleep quality and sleep problems, the overall effects were only slightly adjusted by the trim and fill, suggesting that the impact of the publication bias on the overall effect was rather modest.

Limitations of the Included Studies

A limitation of the included studies is that most were cross-sectional. The lack of longitudinal studies makes it impossible to draw inferences on the directionality between problematic gaming and sleep or to speculate about causality. Moreover, only 13 of the 34 included studies adjusted their estimates for confounding factors (see Table 3). Importantly, all except for one study assessed sleep based upon participants' self-report, potentially introducing response bias such as socially desirable responding and the common method bias.

No study used polysomnography, thus there is a paucity of knowledge on the distribution of sleep stages in problematic gamers. Most of the sleep-related instruments used in research, like the Bergen Insomnia Scale (101), Epworth Sleepiness Scale (102), and the PSQI (103), have been validated against polysomnography in previous studies. In terms of instruments used for the assessment of problematic gaming, these seem to a very limited extent to have been validated against clinical or neurobiological data in previous research (104, 105). Although some of the instruments used for assessing sleep and problematic gaming have been validated against objective or clinical data, it is striking that only a few of the studies included in the present meta-analysis employed such data/assessment (41, 77). Further, most of the studies fail to provide a detailed account regarding the nature of the gaming behavior and sleep problems. For instance, none of the studies registered the time of day the gaming took place. Also, few studies differentiated between sleep data from weekdays and weekends/holidays. Further, few studies registered data on the specific game genres that were played. Consequently, little is known whether some game types are stronger associated with sleep impairments than others. In addition, none of the studies registered whether gaming before sleep was motivated by relaxation or excitement, which would affect arousal and consequently sleep latency. Of the 34 included studies, only two used random population sampling (56, 79), while the remaining used convenience sampling. Moreover, the samples were primarily western-European adolescents. Consequently, the findings might not be generalizable beyond these populations. Lastly, only 10 of the 34 studies were large with more than 1,000 participants, which may put restrictions on the analytic power and the precision of their estimates. Taken together, the abovementioned issues suggest that the available literature has some serious limitations which should be addressed in future studies.

Limitations and Strengths of the Current Review

A limitation of this review is the variability in study populations and spread of operationalizations, measurement tools, and cutoff values to indicate both problematic gaming and sleep problems; suggesting that the current review may have some limitations in terms of comparability between the included studies. This issue can be highlighted by the high levels of heterogeneity found in all the current meta-analyses. In an effort to reduce the heterogeneity, the review could have implemented more stringent inclusion criteria such as only including studies that conceptualized problematic gaming using the four “core addiction” criteria [i.e., conflict, withdrawal, relapse, and problems; cf., (5)] or only included studies adapting the IGD/GD conceptualization. This would, however, further reduce the somewhat restricted pool of studies on the topic and result in insufficient analytic power. Another limitation is the fact that the subgroups of problematic gamers were collapsed in the meta-analyses, although it is conceivable that problematic gamers with more severe symptoms experience more problems with sleep than those with less severe symptoms. Thus, collapsing subgroups may have downplayed the severity of sleep problems among the most problematic gamers. Furthermore, there are also some potential limitations in the present search strategy and inclusion criteria which could cause bias to the synthesis. While the current inclusion criteria did not exclude non-journal manuscripts, the current search strategy makes it likely that scientific work not published in scientific journals could have been overlooked – resulting in possible selection bias. To reduce the risk of bias, the search strategy could have been extended to include gray literature databases (e.g., ProQuest Dissertations & Theses, OpenGrey) and would further have benefited from being validated by a librarian and/or through peer review. Moreover, there might be language bias to the synthesis as the current inclusion criteria excluded all manuscripts that were not in a European language, especially considering the high attention surrounding problematic gaming in Southeast Asian countries (14).

Despite these abovementioned limitations, a strength of the current review is the implementation of two independent reviewers in the screening, data extraction, and study quality assessment processes; reducing the risk of bias and human error and consequently increasing the reliability, reproducibility, and internal validity of the synthesis. Moreover, the current study provides the most comprehensive systematic review on the topic to date and is the first study to quantify the relationship between problematic gaming and multiple sleep-related outcomes using a meta-analytic approach.

Implications for Further Research

This review highlights several future research avenues. The field is in need of more longitudinal studies in order to assess the directionality of the gaming-sleep problems relationship. Moreover, more knowledge is needed on the possible causal pathways involved regarding the relationship between problematic gaming, sleep, and associated outcomes. For instance, the physical, psychological, and functional impairments associated with problematic gaming may reflect an indirect effect between problematic gaming and sleep interference rather than there being a direct causal effect between problematic gaming and negative outcomes. Followingly, future research would benefit from registering more detailed data, such as time of day of gaming, type of game played, sleep both during weekdays and weekends, and specify the nature of the sleep problem as this could provide a better understanding of the mechanisms underlying the association between problematic gaming and sleep. Furthermore, this systematic review and meta-analysis note a potential methodical issue regarding the use of single-item sleep measurement which could yield unprecise estimates. Further research would benefit from assessing sleep parameters using standardized and validated questionnaires as these have less measurement error and would make for better comparisons as well as providing more information on the severity of sleep problems. Also, rather than retrospective questionnaires, further studies would benefit from using sleep diaries as it provides a more detailed picture on the day-to-day relationship between gaming and sleep (e.g., mini-longitudinal design), providing data less prone to recall bias. Moreover, further research would benefit by measuring sleep objectively using polysomnography or actigraphy and to employ clinical screening tools for sleep-related disorders in order to advance the literature. Also, the current evidence base is in need of more nationally representative, and cross-cultural studies, as the currently available data are encumbered with limitations in terms of generalizability.

Conclusion

The current review contributes to the debate on problematic gaming by offering the first quantitative systematic review on problematic gaming and sleep. Overall, the current synthesis suggests that problematic gaming is associated with sleep impairments, confirming that problematic gaming is related to adverse outcomes. The field is, however, currently in urgent need of more high-quality studies and longitudinal investigations. Insight into sleep's role in problematic gaming behavior is important as it could contribute to a better understanding of the etiology of both problematic gaming and sleep problems, which is important for effective assessments, treatments, and health promotion strategies.

Statements

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Author contributions

JK, EE, and SP conceived and designed the study. JK conducted the literature search and wrote the first draft of the manuscript. JK and EE conducted the data extraction and quality assessment. Statistical analyses were conducted by JK under the supervision of SP. All authors contributed to and have approved the final manuscript.

Acknowledgments

The authors would like to thank research assistant Nina Jensen for her contribution to the study screening process.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Footnotes

1.^References marked with an asterisk (*) denote studies included in the current review.

References

  • 1.

    Interactive Software Federation of Europe. Gametrack Survey 2018. (2019). Available online at: https://www.isfe.eu/wp-content/uploads/2019/08/ISFE-Key-Facts-Brochure-FINAL.pdf (accessed March 2, 2021).

  • 2.

    GranicILobelAEngelsRC. The benefits of playing video games. Am Psychol. (2014) 69:6678. 10.1037/a0034857

  • 3.

    AdachiPJCWilloughbyT. More than just fun and games: the longitudinal relationships between strategic video games, self-reported problem solving skills, academic grades. J Youth Adolesc. (2013) 42:104152. 10.1007/s10964-013-9913-9

  • 4.

    KussDJGriffithsMD. Internet gaming addiction: a systematic review of empirical research. Int J Ment Health Addict. (2012) 10:27896. 10.1007/s11469-011-9318-5

  • 5.

    MännikköNRuotsalainenHMiettunenJPontesHMKääriäinenM. Problematic gaming behaviour and health-related outcomes: a systematic review and meta-analysis. J Health Psychol. (2017) 25:6781. 10.1177/1359105317740414

  • 6.

    GentileDAChooHLiauASimTLiDFungDet al. Pathological video game use among youths: a two-year longitudinal study. Pediatrics. (2011) 127:e319. 10.1542/peds.2010-1353

  • 7.

    KrossbakkenEPallesenSMentzoniRAKingDLMoldeHFinseråsTRet al. A cross-lagged study of developmental trajectories of video game engagement, addiction, mental health. Front Psychol. (2018) 9:2239. 10.3389/fpsyg.2018.02239

  • 8.

    KimNRHwangSS-HChoiJ-SKimD-JDemetrovicsZKirályOet al. Characteristics and psychiatric symptoms of internet gaming disorder among adults using self-reported dsm-5 criteria. Psychiatry Invest. (2016) 13:5866. 10.4306/pi.2016.13.1.58

  • 9.

    BrunborgGSMentzoniRAFrøylandLR. Is video gaming, or video game addiction, associated with depression, academic achievement, heavy episodic drinking, or conduct problems?J Behav Addict. (2014) 3:2732. 10.1556/JBA.3.2014.002

  • 10.

    MentzoniRBrunborgGMoldeHMyrsethHSkouverøeKHetlandJet al. Problematic video game use: estimated prevalence and associations with mental and physical health. Cyberpsychol Behav Soc Netw. (2011) 14:5916. 10.1089/cyber.2010.0260

  • 11.

    LemmensJSValkenburgPMPeterJ. Development and validation of a game addiction scale for adolescents. Media Psychol. (2009) 12:7795. 10.1080/15213260802669458

  • 12.

    American Psychological Association. Diagnostic and Statistical Manual of Mental Disorders. Washington, DC: American Psychiatric Publishing (2013). 10.1176/appi.books.9780890425596

  • 13.

    World Health Organization. International Classification of Diseases for Mortality and Morbidity Statistics. Geneva: WHO (2018).

  • 14.

    GriffithsMKirályOPontesHDemetrovicsZ. An overview of problematic gaming. In: AboujaoudeEStarcevicV, editors. Mental Health in the Digital Age: Grave Dangers, Great Promise. Oxford: Oxford University Press (2015). p. 2745. 10.1093/med/9780199380183.003.0002

  • 15.

    FamJY. Prevalence of internet gaming disorder in adolescents: a meta-analysis across three decades. Scand J Psychol. (2018) 59:52431. 10.1111/sjop.12459

  • 16.

    StevensMWDorstynDDelfabbroPHKingDL. Global prevalence of gaming disorder: a systematic review and meta-analysis. Aust N Z J Psychiatry. (2020). 10.1177/0004867420962851. [Epub ahead of print].

  • 17.

    MiharaSHiguchiS. Cross-sectional and longitudinal epidemiological studies of internet gaming disorder: a systematic review of the literature. Psychiatry Clin Neurosci. (2017) 71:42544. 10.1111/pcn.12532

  • 18.

    Van RooijAJFergusonCJCarrasMCKardefelt-WintherDShiJAarsethEet al. A weak scientific basis for gaming disorder: let us err on the side of caution. J Behav Addict. (2018) 7:1. 10.1556/2006.7.2018.19

  • 19.

    AarsethEBeanAMBoonenHColder CarrasMCoulsonMDasDet al. Scholars' open debate paper on the world health organization ICD-11 gaming disorder proposal. J Behav Addict. (2017) 6:26770. 10.1556/2006.5.2016.088

  • 20.

    CarrasMCKardefelt-WintherD. When addiction symptoms and life problems diverge: a latent class analysis of problematic gaming in a representative multinational sample of european adolescents. Eur Child Adolesc Psychiatry. (2018) 27:51325. 10.1007/s00787-018-1108-1

  • 21.

    PallesenSSivertsenBNordhusIHBjorvatnB. A 10-year trend of insomnia prevalence in the adult norwegian population. Sleep Med. (2014) 15:1739. 10.1016/j.sleep.2013.10.009

  • 22.

    LamLT. Internet gaming addiction, problematic use of the internet, and sleep problems: a systematic review. Curr Psychiatry Rep. (2014) 16:444. 10.1007/s11920-014-0444-1

  • 23.

    *HawiNSSamahaMGriffithsMD. Internet gaming disorder in lebanon: relationships with age, sleep habits, academic achievement. J Behav Addict. (2018) 7:708. 10.1556/2006.7.2018.16

  • 24.

    *AltintasEKaracaYHullaertTTassiP. Sleep quality and video game playing: effect of intensity of video game playing and mental health. Psychiatry Res. (2019) 273:48792. 10.1016/j.psychres.2019.01.030

  • 25.

    CainNGradisarM. Electronic media use and sleep in school-aged children and adolescents: a review. Sleep Med. (2010) 11:73542. 10.1016/j.sleep.2010.02.006

  • 26.

    Van den BulckJ. Is television bad for your health? Behavior and body image of the adolescent “Couch Potato”. J Youth Adolesc. (2000) 29:27388. 10.1023/A:1005102523848

  • 27.

    ExelmansLVan den BulckJ. Bedtime, shuteye time and electronic media: sleep displacement is a two-step process. J Sleep Res. (2017) 26:36470. 10.1111/jsr.12510

  • 28.

    LockleySWFosterRG. Sleep: A Very Short Introduction. New York, NY: Oxford University Press (2012). 10.1093/actrade/9780199587858.001.0001

  • 29.

    DijkD-JLockleySW. Invited review: integration of human sleep-wake regulation and circadian rhythmicity. J Appl Physiol. (2002) 92:85262. 10.1152/japplphysiol.00924.2001

  • 30.

    DijkD-JArcherSN. Light, sleep, and circadian rhythms: together again. PLoS Biol. (2009) 7:e1000145. 10.1371/journal.pbio.1000145

  • 31.

    ThoméeSDellveLHärenstamAHagbergM. Perceived connections between information and communication technology use and mental symptoms among young adults-a qualitative study. BMC Public Health. (2010) 10:66. 10.1186/1471-2458-10-66

  • 32.

    LoughranSPWoodAWBartonJMCroftRJThompsonBStoughC. The effect of electromagnetic fields emitted by mobile phones on human sleep. Neuroreport. (2005) 16:19736. 10.1097/01.wnr.0000186593.79705.3c

  • 33.

    LowdenAÅkerstedtTIngreMWiholmCHillertL. Kuster N, et al. Sleep after mobile phone exposure in subjects with mobile phone-related symptoms. Bioelectromagnetics. (2011) 32:4. 10.1002/bem.20609

  • 34.

    KarasekMWoldanska-OkonskaMCzernickiJZylinskaKSwietoslawskiJ. Chronic exposure to 2.9 mt, 40 hz magnetic field reduces melatonin concentrations in humans. J Pineal Res. (1998) 25:2404. 10.1111/j.1600-079X.1998.tb00393.x

  • 35.

    BaglioniCBattaglieseGFeigeBSpiegelhalderKNissenCVoderholzerUet al. Insomnia as a predictor of depression: a meta-analytic evaluation of longitudinal epidemiological studies. J Affect Disord. (2011) 135:109. 10.1016/j.jad.2011.01.011

  • 36.

    MoherDLiberatiATetzlaffJAltmanDGPrisma Group. Preferred reporting items for systematic reviews and meta-analyses: the prisma statement. PLoS Med. (2009) 6:e1000097. 10.1371/journal.pmed.1000097

  • 37.

    PageMJMcKenzieJEBossuytPMBoutronIHoffmannTCMulrowCDet al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. (2021) 372:n71. 10.1136/bmj.n71

  • 38.

    BorensteinMHedgesLVHigginsJPRothsteinHR. Introduction to Meta-Analysis. Chichester: John Wiley and Sons, Ltd (2009). 10.1002/9780470743386

  • 39.

    GiangHTNAhmedAMFalaRYKhattabMMOthmanMHAAbdelrahmanSAMet al. Methodological steps used by authors of systematic reviews and meta-analyses of clinical trials: a cross-sectional study. BMC Med Res Methodol. (2019) 19:164. 10.1186/s12874-019-0780-2

  • 40.

    *LinCYPotenzaMNBrostromAPakpourAH. Internet gaming disorder, psychological distress, and insomnia in adolescent students and their siblings: an actor-partner interdependence model approach. Addict Behav Rep. (2021) 13:100332. 10.1016/j.abrep.2020.100332

  • 41.

    *KoC-HLinH-CLinP-CYenJ-Y. Validity, functional impairment and complications related to internet gaming disorder in the dsm-5 and gaming disorder in the icd-11. Aust N Zeal J Psychiatry. (2019) 54:7078. 10.1177/0004867419881499

  • 42.

    WellsGSheaBO'ConnellDPetersonJWelchVLososMet al. The Newcastle-Ottawa scale (NOS) for Assessing the Quality of Nonrandomised Studies in Meta-Analyses. (2000). Available online at: http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp (accessed May 12, 2021).

  • 43.

    ModestiPAReboldiGCappuccioFPAgyemangCRemuzziGRapiSet al. Panethnic differences in blood pressure in europe: a systematic review and meta-analysis. PLoS ONE. (2016) 11:e0147601. 10.1371/journal.pone.0147601

  • 44.

    KingDLHaagsmaMCDelfabbroPHGradisarMGriffithsMD. Toward a consensus definition of pathological video-gaming: a systematic review of psychometric assessment tools. Clin Psychol Rev. (2013) 33:33142. 10.1016/j.cpr.2013.01.002

  • 45.

    HaleLGuanS. Screen time and sleep among school-aged children and adolescents: a systematic literature review. Sleep Med Rev. (2015) 21:508. 10.1016/j.smrv.2014.07.007

  • 46.

    HigginsJPThompsonSGDeeksJJAltmanDG. Measuring inconsistency in meta-analyses. BMJ. (2003) 327:55760. 10.1136/bmj.327.7414.557

  • 47.

    OrwinRG. A fail-safe n for effect size in meta-analysis. J Educ Stat. (1983) 8:1579. 10.2307/1164923

  • 48.

    DuvalSTweedieR. A nonparametric “trim and fill” method of accounting for publication bias in meta-analysis. J Am Stat Assoc. (2000) 95:8998. 10.1080/01621459.2000.10473905

  • 49.

    DuvalSTweedieR. Trim and fill: a simple funnel-plot–based method of testing and adjusting for publication bias in meta-analysis. Biometrics. (2000) 56:45563. 10.1111/j.0006-341X.2000.00455.x

  • 50.

    CohenJ. Statistical Power Analysis for the Behavioral Sciences. Hillsdale, NJ: Erlbaum (1988).

  • 51.

    FergusonCJ. An effect size primer: a guide for clinicians and researchers. In: KazdinA, editor. Methodological Issues and Strategies in Clinical Research. 4th ed. American Psychological Association (2016). p. 30110. 10.1037/14805-020

  • 52.

    *AchabSNicolierMMaunyFMonninJTrojakBVandelPet al. Massively multiplayer online role-playing games: comparing characteristics of addict vs non-addict online recruited gamers in a french adult population. BMC Psychiatry. (2011) 11:144. 10.1186/1471-244X-11-144

  • 53.

    *AkçayDAkçayBD. The effect of computer game playing habits of university students on their sleep states. Perspect Psychiatr Care. (2020) 12:17. 10.1111/ppc.12497

  • 54.

    *Al AsqahMIAl OraineyAIShukrMAAl OrainiHMAl TurkiYA. The prevalence of internet gaming disorder among medical students at king saud university, riyadh, saudi arabia. A cross-sectional study. Saudi Med J. (2020) 41:135963. 10.15537/smj.2020.12.05584

  • 55.

    *Al GammalMAAli ElsheikhMMAbozahraAA. Internet addiction and internet gaming disorder and associated insomnia among a sample of Al-Azhar university students, clinical study. Egypt J Hospital Med. (2019) 77:571826. 10.21608/ejhm.2019.63227

  • 56.

    *ArnesenAA. Video game addiction among young adults in Norway: prevalence and health (Master thesis). Bergen: University of Bergen (2010).

  • 57.

    *BonnaireCPhanO. [Negative perceptions of the risks associated with gaming in young adolescents: an exploratory study to help thinking about a prevention program]. Arch Pediatrie. (2017) 24:60717. 10.1016/j.arcped.2017.04.006

  • 58.

    *BrunborgGSMentzoniRAMelkevikORTorsheimTSamdalOHetlandJ. Gaming addiction, gaming engagement, and psychological health complaints among norwegian adolescents. Media Psychol. (2013) 16:11528. 10.1080/15213269.2012.756374

  • 59.

    *De PasqualeCSciaccaFMartinelliVChiappediMDinaroCHichyZ. Relationship of internet gaming disorder with psychopathology and social adaptation in italian young adults. Int J Environ Res Public Health. (2020) 17:8201. 10.3390/ijerph17218201

  • 60.

    *FazeliSMohammadi ZeidiILinC-YNamdarPGriffithsMDet al. Depression, anxiety, and stress mediate the associations between internet gaming disorder, insomnia, and quality of life during the Covid-19 outbreak. Addict Behav Rep. (2020) 12:100307. 10.1016/j.abrep.2020.100307

  • 61.

    *FernandesBBiswasUNTan-MansukhaniRVallejoAEssauCA. The impact of covid-19 lockdown on internet use and escapism in adolescents. Rev Psicol Clin Ninos Adolesc. (2020) 7:5965. 10.21134/rpcna.2020.mon.2056

  • 62.

    *Gonzalez-ValeroGLuis Ubago-JimenezJAndres Ramirez-GranizIPuertas-MoleroP. Relationship between the use of video games and physical-healthy, psychosocial and academic indicators in primary schoolchildren. J Hum Sport Exerc. (2020) 15:S33644. 10.14198/jhse.2020.15.Proc2.25

  • 63.

    *KimNHughesTLParkCGQuinnLKongID. Resting-state peripheral catecholamine and anxiety levels in korean male adolescents with internet game addiction. Cyberpsychol Behav Soc Netw. (2016) 19:2028. 10.1089/cyber.2015.0411

  • 64.

    *KingDLDelfabbroPHZwaansTKaptsisD. Sleep interference effects of pathological electronic media use during adolescence. Int J Mental Health Addict. (2014) 12:2135. 10.1007/s11469-013-9461-2

  • 65.

    *LinC-YImaniVBrostromAArestedtKPakpourAHet al. Evaluating the psychometric properties of the 7-item persian game addiction scale for iranian adolescents. Front Psychol. (2019) 10:149. 10.3389/fpsyg.2019.00149

  • 66.

    *LiuYWangQJouMWangBAnYLiZ. Psychometric properties and measurement invariance of the 7-item game addiction scale (GAS) among chinese college students. BMC Psychiatry. (2020) 20:484. 10.1186/s12888-020-02830-7

  • 67.

    *MännikköNBillieuxJKaariainenM. Problematic digital gaming behavior and its relation to the psychological, social and physical health of finnish adolescents and young adults. J Behav Addict. (2015) 4:2818. 10.1556/2006.4.2015.040

  • 68.

    *MännikköNRuotsalainenHTolvanenAKääriäinenM. Problematic gaming is associated with some health-related behaviors among finnish vocational school students. Int J Ment Health Addict. (2019) 18:9931007. 10.1007/s11469-019-00100-6

  • 69.

    *NakayamaHMatsuzakiTMiharaSKitayuguchiTHiguchiS. Relationship between problematic gaming and age at the onset of habitual gaming. Pediatrics Int. (2020) 62:127581. 10.1111/ped.14290

  • 70.

    *NogueiraMFariaHVitorinoASilvaFGNetoAS. Addictive video game use: an emerging pediatric problem?Acta Med Port. (2019) 32:1838. 10.20344/amp.10985

  • 71.

    *PeracchiaSTribertiSCurcioG. Longer the game, better the sleep: intense video game playing is associated to better sleep quality and better daytime functioning. Ann Rev CyberTherapy Telemed. (2017) 15:2046.

  • 72.

    *RehbeinFKleimannMMossleT. Prevalence and risk factors of video game dependency in adolescence: results of a German nationwide survey. Cyberpsychol Behav Soc Netw. (2010) 13:26977. 10.1089/cyber.2009.0227

  • 73.

    *RehbeinFKliemSBaierDMosleTPetryNM. Prevalence of internet gaming disorder in german adolescents: diagnostic contribution of the nine DSM-5 criteria in a state-wide representative sample. Addiction. (2015) 110:84251. 10.1111/add.12849

  • 74.

    *SatgharePAbdinEVaingankarJAChuaBYPangSPiccoLet al. Prevalence of sleep problems among those with internet gaming disorder in singapore. Asian J Psychiatry. (2016) 17:18898.

  • 75.

    *SeveroRBSoaresJMAffonsoJPGiustiDAde Souza JuniorAAde FigueiredoVLet al. Prevalence and risk factors for internet gaming disorder. Braz J Psychiatry. (2020) 42:5325. 10.1590/1516-4446-2019-0760

  • 76.

    *StockdaleLCoyneSM. Video game addiction in emerging adulthood: cross-sectional evidence of pathology in video game addicts as compared to matched healthy controls. J Affect Disord. (2018) 225:26572. 10.1016/j.jad.2017.08.045

  • 77.

    *TurelORomashkinAMorrisonKM. Health outcomes of information system use lifestyles among adolescents: videogame addiction, sleep curtailment and cardio-metabolic deficiencies. PLoS ONE. (2016) 11:e0154764. 10.1371/journal.pone.0154764

  • 78.

    *VollmerCRandlerCHorzumMBAyasT. Computer game addiction in adolescents and its relationship to chronotype and personality. SAGE Open. (2014) 4:19. 10.1177/2158244013518054

  • 79.

    *WenzelHBakkenIJohanssonAGotestamKOrenA. Excessive computer game playing among norwegian adults: self-reported consequences of playing and association with mental health problems. Psychol Rep. (2009) 105:123747. 10.2466/PR0.105.F.1237-1247

  • 80.

    *WongHYMoHYPotenzaMNChanMNMLauWMChuiTKet al. Relationships between severity of internet gaming disorder, severity of problematic social media use, sleep quality and psychological distress. Int J Environ Res Public Health. (2020) 17:1879. 10.3390/ijerph17061879

  • 81.

    *YuYYangXWangSWangHChangRTsamlagLet al. Serial multiple mediation of the association between internet gaming disorder and suicidal ideation by insomnia and depression in adolescents in shanghai, China. BMC Psychiatry. (2020) 20:460. 10.1186/s12888-020-02870-z

  • 82.

    ChenHCohenPChenS. How big is a big odds ratio? Interpreting the magnitudes of odds ratios in epidemiological studies. Commun Stat Simulat Comput. (2010) 39:8604. 10.1080/03610911003650383

  • 83.

    NemesSJonassonJMGenellASteineckG. Bias in odds ratios by logistic regression modelling and sample size. BMC Med Res Methodol. (2009) 9:56. 10.1186/1471-2288-9-56

  • 84.

    BreaughJA. Effect size estimation: factors to consider and mistakes to avoid. J Manag. (2003) 29:7997. 10.1177/014920630302900106

  • 85.

    PrenticeDAMillerDT. When small effects are impressive. Psychol Bull. (1992) 112:1604. 10.1037/0033-2909.112.1.160

  • 86.

    Endocrine Society. Losing 30 Minutes of Sleep Per Day May Promote Weight Gain and Adversely Affect Blood Sugar Control. ScienceDaily (2015). Available online at: www.sciencedaily.com/releases/2015/03/150306082541.htm (accessed March 02, 2021).

  • 87.

    AlimoradiZLinC-YBroströmABülowPHBajalanZet al. Internet addiction and sleep problems: a systematic review and meta-analysis. Sleep Med Rev. (2019) 47:5161. 10.1016/j.smrv.2019.06.004

  • 88.

    SohnSReesPWildridgeBKalkNJCarterB. Prevalence of problematic smartphone usage and associated mental health outcomes amongst children and young people: a systematic review, meta-analysis and grade of the evidence. BMC Psychiatry. (2019) 19:356. 10.1186/s12888-019-2350-x

  • 89.

    MeiXZhouQLiXJingPWangXHuZ. Sleep problems in excessive technology use among adolescent: a systemic review and meta-analysis. Sleep Sci Pract. (2018) 2:9. 10.1186/s41606-018-0028-9

  • 90.

    DworakMSchierlTBrunsTStrüderHK. Impact of singular excessive computer game and television exposure on sleep patterns and memory performance of school-aged children. Pediatrics. (2007) 120:978. 10.1542/peds.2007-0476

  • 91.

    GradisarMWolfsonARHarveyAGHaleLRosenbergRCzeislerCA. The sleep and technology use of americans: findings from the national sleep foundation's 2011 sleep in america poll. J Clin Sleep Med. (2013) 9:12919. 10.5664/jcsm.3272

  • 92.

    ExelmansLVan den BulckJ. Sleep research: a primer for media scholars. Health Commun. (2019) 34:51928. 10.1080/10410236.2017.1422100

  • 93.

    CarskadonMAVieiraCAceboC. Association between puberty and delayed phase preference. Sleep. (1993) 16:25862. 10.1093/sleep/16.3.258

  • 94.

    PaineS-JGanderPHTravierN. The epidemiology of morningness/eveningness: influence of age, gender, ethnicity, and socioeconomic factors in adults (30-49 years). J Biol Rhythms. (2006) 21:6876. 10.1177/07487304052831541

  • 95.

    CrowleySJAceboCCarskadonMA. Sleep, circadian rhythms, and delayed phase in adolescence. Sleep Med. (2007) 8:60212. 10.1016/j.sleep.2006.12.002

  • 96.

    BartelKAGradisarMWilliamsonP. Protective and risk factors for adolescent sleep: a meta-analytic review. Sleep Med Rev. (2015) 21:7285. 10.1016/j.smrv.2014.08.002

  • 97.

    HysingMHarveyAGBøeTHeradstveitOVedaaØSivertsenB. Trajectories of sleep problems from adolescence to adulthood. Linking two population-based studies from Norway. Sleep Med. (2020) 75:4117. 10.1016/j.sleep.2020.08.035

  • 98.

    DiamantopoulosASarstedtMFuchsCWilczynskiPKaiserS. Guidelines for choosing between multi-item and single-item scales for construct measurement: a predictive validity perspective. J Acad Market Sci. (2012) 40:43449. 10.1007/s11747-011-0300-3

  • 99.

    LallukkaTDreganAArmstrongD. Comparison of a sleep item from the general health questionnaire-12 with the jenkins sleep questionnaire as measures of sleep disturbance. J Epidemiol. (2011) 21:47480. 10.2188/jea.JE20110023

  • 100.

    SongFHooperLLokeY. Publication bias: what is it? How do we measure it? How do we avoid it?Open Access J Clin Trials. (2013) 5:7181. 10.2147/OAJCT.S34419

  • 101.

    PallesenSBjorvatnBNordhusIHSivertsenBHjørnevikMMorinCM. A new scale for measuring insomnia: the bergen insomnia scale. Percept Mot Skills. (2008) 107:691706. 10.2466/pms.107.3.691-706

  • 102.

    ChervinRDAldrichMSPickettRChristianG. Comparison of the results of the epworth sleepiness scale and the multiple sleep latency test. J Psychosom Res. (1997) 42:14555. 10.1016/S0022-3999(96)00239-5

  • 103.

    BackhausJJunghannsKBroocksARiemannDHohagenF. Test–retest reliability and validity of the pittsburgh sleep quality index in primary insomnia. J Psychosom Res. (2002) 53:73740. 10.1016/S0022-3999(02)00330-6

  • 104.

    KingDLChamberlainSRCarragherNBillieuxJSteinDMuellerKet al. Screening and assessment tools for gaming disorder: a comprehensive systematic review. Clin Psychol Rev. (2020) 77:101831. 10.1016/j.cpr.2020.101831

  • 105.

    KayeLKKussDJRumpfH-J. Conceptual and methodological considerations of gaming disorder and internet gaming disorder. In: El-GuebalyNCarràGGalanterMBaldacchinoAM, editors. Textbook of Addiction Treatment: International Perspectives.Cham: Springer International Publishing (2021). p. 96777. 10.1007/978-3-030-36391-8_68

Summary

Keywords

gaming addiction, gaming disorder, internet gaming disorder, sleep, sleep problems, insomnia, systematic review, meta-analysis

Citation

Kristensen JH, Pallesen S, King DL, Hysing M and Erevik EK (2021) Problematic Gaming and Sleep: A Systematic Review and Meta-Analysis. Front. Psychiatry 12:675237. doi: 10.3389/fpsyt.2021.675237

Received

02 March 2021

Accepted

04 May 2021

Published

07 June 2021

Volume

12 - 2021

Edited by

Nitya Jayaram-Lindstrom, Karolinska Institutet (KI), Sweden

Reviewed by

Kristoffer Magnusson, Karolinska Institutet, Sweden; Georgios Demetrios Kotzalidis, Sapienza University of Rome, Italy

Updates

Copyright

*Correspondence: Joakim H. Kristensen

This article was submitted to Addictive Disorders, a section of the journal Frontiers in Psychiatry

Disclaimer

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

Outline

Figures

Cite article

Copy to clipboard


Export citation file


Share article

Article metrics