Edited by: Jeanne Frances Duffy, Brigham and Women’s Hospital and Harvard Medical School, United States
Reviewed by: Melissa St. Hilaire, Brigham and Women’s Hospital and Harvard Medical School, United States; Andrew J. K. Phillips, Monash University, Australia
This article was submitted to Sleep and Circadian Rhythms, a section of the journal Frontiers in Neuroscience
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
The Psychomotor Vigilance Test (PVT) is a widely used behavioral attention measure, with the 10-min (PVT-10) and 3-min (PVT-3) as two commonly used versions. The PVT-3 may be comparable to the PVT-10, though its convergent validity relative to the PVT-10 has not been explicitly assessed. For the first time, we utilized repeated measures correlation (rmcorr) to evaluate intra-individual associations between PVT-10 and PVT-3 versions across total sleep deprivation (TSD), chronic sleep restriction (SR) and multiple consecutive days of recovery. Eighty-three healthy adults (mean ± SD, 34.7 ± 8.9 years; 36 females) received two baseline nights (B1-B2), five SR nights (SR1-SR5), 36 h TSD, and four recovery nights (R1-R4) between sleep loss conditions. The PVT-10 and PVT-3 were completed every 2 h during wakefulness. Rmcorr compared responses on two frequently used, sensitive PVT metrics: reaction time (RT)
One of the most commonly utilized measures in sleep research is the Psychomotor Vigilance Test (PVT), a measure of vigilant attention that requires participants to rapidly respond to visual cues randomly presented within specified interstimulus intervals (ISIs) without incorrectly responding when no stimulus is present (
Two published studies have directly compared performance on the PVT-10 and PVT-3 in response to sleep loss without using any other experimental manipulations: (1) the PVT-3 development study (
Though averaging data from multiple time points may be necessary to meet various statistical assumptions, doing so can result in the loss of important data relating to changes in performance across time. Of note, the aforementioned sleep loss studies comparing the PVT-10 and PVT-3 (
Little is known about PVT performance across extended recovery periods (e.g., more than one consecutive recovery night) following sleep deprivation (
Given that prior studies found significant differences between the PVT-3 and PVT-10, that no sleep loss studies administered both versions on the same device or included an extended recovery period, that most analyses utilized averaged data, and that the PVT-3 is increasingly utilized (
The current study utilized the repeated measures correlation (rmcorr) technique (
Eighty-three healthy adults were recruited in response to study advertisements. Participants reported habitual nightly sleep durations between 6.5 and 8.5 h, with habitual bedtimes between 2200 and 0000 h, and habitual awakenings between 0600 and 0930 h; these were confirmed
Participants engaged in a 13-day laboratory study during which they received daily checks of vital signs and symptoms by nurses (with a physician on call). The 13-day study consisted of two baseline nights (B1-B2, 10 h [2200–0800 h] and 12 h [2200–1000 h] TIB, respectively) followed by randomization to either five nights of 4 h TIB SR (SR1-SR5, 0400–0800 h,
A computer-based neurobehavioral test battery was administered every 2 h during wakefulness throughout the study. Between test bouts participants were ambulatory and permitted to perform sedentary activities; however, they were not allowed to exercise. Ambient temperature was maintained between 22 and 24°C. Laboratory light levels remained constant at <50 lux during scheduled wakefulness and <1 lux during scheduled sleep periods (
The computer-based neurobehavioral test battery included two widely used versions of a measure of behavioral attention: the 10-min PVT (
Although repeated measures data are inherently valuable, their analyses can be challenging due to frequent violation of the assumptions of various statistical procedures (
Rmcorr confidence intervals (CIs) were determined using bootstrapping with replacement and using 1,000 samples (
PVT-10 and PVT-3 lapses rmcorr results by day and by study phase.
Condition | Statistic | Study day |
Study phase |
||||||||||||||
B2 | SR1 | SR2 | SR3 | SR4 | SR5 | R1 | R2 | R3 | R4 | TSD | ALL | B2-R4 | SR1-SR5 | R1-R4 | TSD | ||
A | 0.134 | 0.395 | 0.430 | 0.388 | 0.420 | 0.398 | 0.523 | 0.178 | 0.145 | 0.293 | 0.489 | 0.566 | 0.535 | 0.472 | 0.334 | 0.489 | |
156 | 327 | 368 | 366 | 368 | 182 | 140 | 202 | 162 | 163 | 409 | 3345 | 2894 | 1779 | 793 | 409 | ||
0.093 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | 0.011 | 0.065 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | ||
CI low | −0.005 | 0.299 | 0.315 | 0.284 | 0.304 | 0.227 | 0.105 | −0.039 | −0.057 | 0.112 | 0.422 | 0.534 | 0.499 | 0.423 | 0.228 | 0.422 | |
CI high | 0.259 | 0.483 | 0.547 | 0.489 | 0.528 | 0.516 | 0.699 | 0.350 | 0.333 | 0.443 | 0.563 | 0.597 | 0.570 | 0.520 | 0.426 | 0.563 | |
PVT-10 | Mdn |
0.00 |
1.00 |
2.00 |
2.00 |
3.00 |
3.00 |
1.00 |
1.00 |
1.00 |
1.00 |
4.00 |
2.00 |
1.00 |
2.00 |
1.00 |
4.00 |
PVT-3 | Mdn |
1.00 |
2.00 |
4.00 |
4.00 |
6.00 |
6.00 |
2.00 |
2.00 |
2.00 |
2.00 |
8.00 |
3.00 |
3.00 |
4.00 |
2.00 |
8.00 |
B | 0.000 | 0.547 | 0.066 | 0.258 | 0.238 | 0.813 | 0.563 | 0.544 | 0.519 | 0.446 | 0.330 | 0.662 | 0.705 | 0.547 | 0.513 | 0.524 | |
149 | 388 | 166 | 209 | 167 | 167 | 335 | 377 | 377 | 377 | 186 | 3442 | 1449 | 388 | 838 | 1824 | ||
1.000 | <0.001 | 0.392 | <0.001 | 0.002 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | ||
CI low | −0.253 | 0.449 | −0.182 | 0.002 | −0.010 | −0.279 | 0.439 | 0.414 | 0.412 | 0.338 | 0.147 | 0.633 | 0.661 | 0.449 | 0.067 | 0.469 | |
CI high | 0.204 | 0.637 | 0.270 | 0.473 | 0.469 | 0.909 | 0.676 | 0.648 | 0.630 | 0.547 | 0.521 | 0.692 | 0.749 | 0.637 | 0.730 | 0.577 | |
PVT-10 | Mdn |
0.00 |
2.00 |
0.00 |
0.00 |
0.00 |
0.00 |
1.00 |
1.00 |
2.00 |
2.00 |
2.00 |
1.00 |
0.00 |
2.00 |
0.00 |
2.00 |
PVT-3 | Mdn |
1.00 |
5.00 |
1.00 |
1.00 |
1.00 |
1.00 |
3.00 |
4.00 |
4.00 |
6.00 |
5.00 |
3.00 |
2.00 |
5.00 |
1.00 |
1.00 |
Eighty-three healthy adults (mean ± SD, 34.7 ± 8.9 years; 36 females) (aged 21–50 years, 72.3% African American; 43.4% female) participated in the study, with
PVT-10 and PVT-3 transformed lapses rmcorr results by time point.
Condition | Statistic | All study days |
Recovery only |
||||||||
Time | 1000 h | 1200 h | 1600 h | 1800 h | 2000 h | 1000 h | 1200 h | 1600 h | 1800 h | 2000 h | |
A | 0.594 | 0.593 | 0.523 | 0.436 | 0.461 | 0.238 | 0.319 | 0.224 | 0.148 | 0.319 | |
387 | 409 | 405 | 408 | 383 | 100 | 122 | 119 | 121 | 122 | ||
<0.001 | <0.001 | <0.001 | <0.001 | <0.001 | 0.016 | <0.001 | 0.014 | 0.103 | <0.001 | ||
CI low | 0.529 | 0.525 | 0.444 | 0.360 | 0.369 | 0.057 | 0.173 | 0.047 | −0.020 | 0.126 | |
CI high | 0.652 | 0.656 | 0.592 | 0.512 | 0.549 | 0.415 | 0.464 | 0.427 | 0.308 | 0.488 | |
PVT-10 |
Mdn |
1.00 |
1.00 |
1.00 |
1.00 |
1.00 |
1.00 |
1.00 |
1.00 |
1.00 |
1.00 |
PVT-3 |
Mdn |
3.00 |
3.00 |
3.00 |
3.00 |
3.00 |
2.00 |
2.00 |
2.00 |
3.00 |
2.00 |
B | 0.589 | 0.514 | 0.502 | 0.506 | 0.527 | 0.199 | 0.031 | 0.125 | 0.082 | 0.242 | |
419 | 418 | 419 | 419 | 389 | 125 | 124 | 125 | 125 | 125 | ||
<0.001 | <0.001 | <0.001 | <0.001 | <0.001 | 0.025 | 0.733 | 0.163 | 0.359 | 0.006 | ||
CI low | 0.523 | 0.438 | 0.433 | 0.430 | 0.445 | 0.042 | −0.175 | −0.036 | −0.136 | 0.094 | |
CI high | 0.645 | 0.577 | 0.569 | 0.574 | 0.595 | 0.346 | 0.225 | 0.299 | 0.263 | 0.391 | |
PVT-10 |
Mdn |
1.00 |
1.00 |
1.00 |
1.00 |
1.00 |
0.00 |
0.00 |
0.00 |
0.00 |
0.00 |
PVT-3 |
Mdn |
2.00 |
2.00 |
2.00 |
2.00 |
2.00 |
1.00 |
1.00 |
1.00 |
1.00 |
1.00 |
Rmcorr plots of repeated-measures correlations between 10-min Psychomotor Vigilance Test (PVT-10) and 3-min Psychomotor Vigilance Test (PVT-3) lapses by study phase for Condition A
Overall, Condition B yielded stronger correlations relative to Condition A, and all the by study phase analyses for PVT lapses were significant (
The by day rmcorr analyses revealed a wide range of rmcorr coefficient values for PVT lapses across study days (
Rmcorr plots of repeated-measures correlations between 10-min Psychomotor Vigilance Test (PVT-10) and 3-min Psychomotor Vigilance Test (PVT-3) lapses by study day for Condition A
The entire study (all-study) duration time point rmcorr analyses for PVT lapses were all significant for Condition A and Condition B (
Rmcorr plots of repeated-measures correlations between 10-min Psychomotor Vigilance Test (PVT-10) and 3-min Psychomotor Vigilance Test (PVT-3) transformed lapses at 1800 h across the entire study (All Study Days) and across only recovery days 1–4 (R1-R4) for Condition A
PVT-10 and PVT-3 1/RT rmcorr results by day and by study phase.
Condition | Statistic | Study day |
Study phase |
||||||||||||||
B2 | SR1 | SR2 | SR3 | SR4 | SR5 | R1 | R2 | R3 | R4 | TSD | ALL | B2-R4 | SR1-SR5 | R1-R4 | TSD | ||
A | 0.284 | 0.699 | 0.629 | 0.623 | 0.590 | 0.482 | 0.540 | 0.310 | 0.375 | 0.352 | 0.691 | 0.712 | 0.678 | 0.663 | 0.422 | 0.691 | |
156 | 327 | 368 | 366 | 368 | 182 | 140 | 202 | 162 | 163 | 409 | 3345 | 2894 | 1779 | 793 | 409 | ||
<0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | ||
CI low | 0.129 | 0.627 | 0.543 | 0.539 | 0.523 | 0.328 | 0.391 | 0.146 | 0.232 | 0.189 | 0.633 | 0.690 | 0.650 | 0.632 | 0.357 | 0.633 | |
CI high | 0.451 | 0.762 | 0.702 | 0.695 | 0.655 | 0.598 | 0.663 | 0.463 | 0.499 | 0.484 | 0.741 | 0.733 | 0.702 | 0.696 | 0.481 | 0.741 | |
PVT-10 | Mdn |
4.03 |
3.70 |
3.60 |
3.50 |
3.36 |
3.29 |
3.65 |
3.63 |
3.64 |
3.62 |
3.21 |
3.54 |
3.58 |
3.49 |
3.63 |
3.21 |
PVT-3 | Mdn |
4.31 |
4.01 |
3.82 |
3.76 |
3.56 |
3.59 |
4.13 |
3.95 |
4.03 |
3.92 |
3.49 |
3.85 |
3.91 |
3.76 |
4.02 |
3.49 |
B | 0.305 | 0.676 | 0.408 | 0.376 | 0.370 | 0.526 | 0.747 | 0.744 | 0.691 | 0.691 | 0.609 | 0.808 | 0.784 | 0.676 | 0.466 | 0.737 | |
149 | 388 | 166 | 209 | 167 | 167 | 335 | 377 | 377 | 377 | 186 | 3442 | 1449 | 388 | 838 | 1824 | ||
<0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | ||
CI low | 0.152 | 0.611 | 0.272 | 0.217 | 0.228 | 0.164 | 0.688 | 0.670 | 0.620 | 0.616 | 0.485 | 0.791 | 0.757 | 0.611 | 0.353 | 0.706 | |
CI high | 0.465 | 0.734 | 0.521 | 0.498 | 0.508 | 0.695 | 0.798 | 0.807 | 0.754 | 0.757 | 0.709 | 0.823 | 0.810 | 0.734 | 0.569 | 0.767 | |
PVT-10 | Mdn |
3.88 |
3.48 |
3.89 |
3.88 |
3.90 |
3.83 |
3.60 |
3.55 |
3.50 |
3.42 |
3.45 |
3.64 |
3.78 |
3.48 |
3.88 |
3.50 |
PVT-3 | Mdn |
4.12 |
3.61 |
4.21 |
4.16 |
4.23 |
4.13 |
3.82 |
3.75 |
3.70 |
3.59 |
3.60 |
3.87 |
4.04 |
3.61 |
4.17 |
3.71 |
PVT-10 and PVT-3 1/RT rmcorr results by time point.
Condition | Statistic | All study days |
Recovery only |
||||||||
Time | 1000 h | 1200 h | 1600 h | 1800 h | 2000 h | 1000 h | 1200 h | 1600 h | 1800 h | 2000 h | |
A | 0.748 | 0.698 | 0.697 | 0.565 | 0.651 | 0.408 | 0.463 | 0.377 | 0.296 | 0.400 | |
387 | 409 | 405 | 408 | 383 | 100 | 122 | 119 | 121 | 122 | ||
<0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | 0.001 | <0.001 | ||
CI low | 0.701 | 0.634 | 0.622 | 0.491 | 0.566 | 0.242 | 0.314 | 0.150 | 0.130 | 0.227 | |
CI high | 0.791 | 0.759 | 0.760 | 0.636 | 0.724 | 0.578 | 0.606 | 0.570 | 0.439 | 0.564 | |
PVT-10 | Mdn |
3.61 |
3.64 |
3.65 |
3.51 |
3.54 |
3.69 |
3.73 |
3.64 |
3.55 |
3.58 |
PVT-3 | Mdn |
3.89 |
3.88 |
3.93 |
3.93 |
3.91 |
4.03 |
4.01 |
4.03 |
3.94 |
4.03 |
B | 0.824 | 0.784 | 0.792 | 0.736 | 0.808 | 0.506 | 0.354 | 0.449 | 0.632 | 0.449 | |
419 | 418 | 419 | 419 | 389 | 125 | 124 | 125 | 125 | 125 | ||
<0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | ||
CI low | 0.782 | 0.740 | 0.749 | 0.682 | 0.766 | 0.356 | 0.173 | 0.265 | 0.290 | 0.321 | |
CI high | 0.860 | 0.822 | 0.827 | 0.788 | 0.845 | 0.652 | 0.519 | 0.589 | 0.772 | 0.586 | |
PVT-10 | Mdn |
3.71 |
3.72 |
3.71 |
3.67 |
3.69 |
3.90 |
3.89 |
3.86 |
3.82 |
3.84 |
PVT-3 | Mdn |
3.91 |
3.94 |
3.95 |
3.97 |
3.96 |
4.15 |
4.17 |
4.20 |
4.17 |
4.12 |
Rmcorr plots of repeated-measures correlations between 10-min Psychomotor Vigilance Test (PVT-10) and 3-min Psychomotor Vigilance Test (PVT-3) response speed (1/RT) by study phase for Condition A
For PVT 1/RT rmcorr analyses across study phases, Condition B generally yielded stronger correlations relative to Condition A, but all by study phase analyses were significant for both conditions (
The by day rmcorr analyses revealed a wide range of
Rmcorr plots of repeated-measures correlations between 10-min Psychomotor Vigilance Test (PVT-10) and 3-min Psychomotor Vigilance Test (PVT-3) response speed (1/RT) by study day for Condition A
The entire study (all-study) duration time point rmcorr analyses for PVT 1/RT were all significant for Condition A and Condition B (
Rmcorr plots of repeated-measures correlations between 10-min Psychomotor Vigilance Test (PVT-10) and 3-min Psychomotor Vigilance Test (PVT-3) response speed (1/RT) at 1800 h across the entire study (All Study Days) and across only recovery days 1–4 (R1-R4) for Condition A
This is the first study examining the convergent validity of the PVT-3 relative to the “gold standard” PVT-10 across two commonly experienced sleep loss types followed by an extended recovery period when administered on the same device. Correlations for PVT 1/RT were stronger relative to PVT lapses throughout the study, yet both metrics were not strongly correlated consistently throughout SR and TSD. Notably, PVT-3 lapses and 1/RT both demonstrated poor correlations with the respective PVT-10 measures during baseline and recovery periods, when participants were not undergoing experimentally induced sleep loss. Generally, the PVT-3 demonstrated inadequate convergent validity (it failed to show
We hypothesized that rmcorr analyses would show relatively strong correlations between the PVT-10 and PVT-3 across all study phases of the sleep deprivation study on two frequently utilized PVT outcome metrics. Contrary to our expectations, only two
Considered across the study, correlations for PVT 1/RT were generally stronger than those for PVT lapses, thus supporting our hypothesis. Our results correspond with previous studies that found PVT lapses more consistently differed and demonstrated lower correlations between the PVT-10 and PVT-3 relative to PVT 1/RT (
Our hypothesis that correlations would be strongest during sleep deprived study phases compared to rested study phases (baseline and recovery) was generally supported. Correlations for recovery only time points, across recovery days, and across the recovery study phase were almost always weaker than those demonstrated during periods of sleep deprivation, while baseline correlations were non-significant, negligible, or weak. Since the
Different ISI durations between PVT versions could have potentially contributed to differential sensitivity to performance during rested and sleep deprived periods. One study failed to find a differential impact on PVT-10 and PVT-3 performance but reported that TSD enhanced the ISI effect (
The fact that the PVT-10 and PVT-3 are not comparable in their ability to measure behavioral attention during rested periods has implications for study design, test selection, and use of biomarkers or predictors relating to performance (
Lastly, our hypothesis that correlation patterns across the extended recovery study phase would not differ between those exposed to TSD relative to those exposed to SR was not supported. Interestingly, correlations between the measures on R1 for those exposed to SR first (Condition A) were moderate while they were non-significant and negligible to weak for those exposed to TSD first (Condition B). Correlations on R2-R3 were comparable between conditions, yet R4 demonstrated negligible to weak correlations for Condition A, but these were moderate-to-strong for Condition B. Given that research on performance throughout extended recovery periods is limited, interpretation of these findings is challenging, but might relate to a differential recovery neurobehavioral performance profile following SR relative to TSD. Indeed, this is in line with a study from
Our study had a few limitations. These findings may not apply to different duration versions of the PVT such as the PVT-5 (
The highly controlled nature of our study, the large sample, the same device administration and the ability to utilize all available study data are unique strengths in the context of an evaluation of convergent validity. In our study, although PVT 1/RT specifically demonstrated relatively strong correlations across SR and TSD, most correlations were below an acceptable threshold for the measures to be considered interchangeable, while such correlations for PVT lapses were consistently below that threshold. Our results, coupled with the discordant results between the PVT-10 and PVT-3 observed during sleep loss on at least one major outcome metric in prior PVT-3 studies (
The data generated and analyzed during the current study are available from the corresponding author upon reasonable request.
The studies involving human participants were reviewed and approved by the University of Pennsylvania’s Institutional Review Board. The participants provided their written informed consent to participate in this study.
NG designed the original study during which the analyzed data were collected and extracted and retains oversight and control of all data as Principal Investigator. NG provided financial support, including all research grant support, all personnel support, and all publication fees. CA conceived of the present study, identified and verified the statistical methods, and performed the analyses. EY, CC, and TB checked analysis scripts, plots, and results for accuracy. CA wrote the manuscript. All authors contributed to the interpretation of the results, provided critical feedback, helped shape the research, analysis, and manuscript, and reviewed and approved the final manuscript.
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
This work was primarily supported by the Department of the Navy, Office of Naval Research (Award No. N00014-11-1-0361) to NG Other support provided by National Aeronautics and Space Administration (NASA) NNX14AN49G and grant 80NSSC20K0243 (to NG), National Institutes of Health grant NIH R01DK117488 (to NG) and Clinical and Translational Research Center grant UL1TR000003. None of the sponsors had any role in the following: design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript.
We thank the faculty and staff of the Unit for Experimental Psychiatry for their contributions to this study in terms of data collection.