RESEARCH ARTICLE
Validation of a Portable, Touch-Screen Psychomotor Vigilance Test Kimberly A. Honn; Samantha M. Riedy; Devon A. Grant
INTRODUCTION:
METHODS:
RESULTS:
DISCUSSION:
KEYWORDS:
The Psychomotor Vigilance Test (PVT) measures effects of fatigue from sleep loss and circadian misalignment on sustained vigilance performance. To promote PVT use in field environments, a 5-min PVT version has been implemented on a personal digital Delivered assistant (PDA) a touch TheUniversity present laboratory study was conducted to validate this by with Ingenta to:screen. Purdue Libraries PVT against a standard laptop PVT across 38 h of03 total sleep deprivation (TSD). IP:10-min 79.110.17.193 On: Wed, Aug 2016 02:31:15
Aerospace Association Following a baseline sleepCopyright: night, subjects underwent Medical 38 h of TSD, during which they performed the PVT every hour, alternating between the two test platforms. The study concluded with a night of recovery sleep. The primary outcome was the number of PVT lapses (reaction times . 500 ms). Both PVT platforms showed significant effects for the number of lapses across TSD test times involving an increase with time awake modulated by circadian rhythm. Laptop PVT lapses across test times exhibited a large effect size (f2 5 0.36), whereas PDA PVT lapses exhibited a medium effect size (f2 5 0.17). The laptop PVT showed a significant effect for the number of false starts during TSD similar to the temporal profile of lapses, while the PDA PVT had false starts throughout the TSD period. The 5-min PDA PVT provided performance testing functionality and results comparable to the 10-min laptop PVT. The number of PDA PVT lapses tracked fatigue similarly to the laptop PVT lapses, albeit with smaller average ranges and effect sizes. sleep deprivation, fatigue, neurobehavioral performance, PVT, lapses of attention. Honn KA, Riedy SM, Grant DA. Validation of a portable, touch-screen psychomotor vigilance test. Aerosp Med Hum Perform. 2015; 86(5):428–434.
T
he psychomotor vigilance test,10,17 commonly referred to by the acronym PVT, is a computerized serial reaction time (RT) test requiring subjects to respond, by means of button presses, to stimuli appearing on a screen at random intervals. The PVT’s requirement for sustained attention (typically for 3, 5, 10, or 20 min per test bout) and its high stimulus density (on the order of about 10 stimuli per minute or more) make the test highly sensitive to fatigue (sleepiness) from sleep loss, circadian misalignment, and sleep disorders.3,16 The PVT records RTs, from which a variety of outcome measures can be computed, such as mean or median RT, 10% fastest and 10% slowest RTs (and transformations thereof), number of lapses (conventionally defined as RTs longer than a 500-ms threshold), and number of false starts (errors of commission). These outcome measures capture different aspects of the underlying RT distribution, which displays a long right-hand tail. Fatigue causes progressively more RTs to end up in the right-hand tail.9,11 This shift can be adequately described by a one-parameter shift in one of the RT
428
AEROSPACE MEDICINE AND HUMAN PERFORMANCE Vol. 86, No. 5
distribution parameters.19 As such, in the context of measuring fatigue, different PVT outcome measures capture essentially the same phenomenon, and selection of a primary outcome measure hinges primarily on signal-to-noise ratio and other practical considerations. The number of lapses, a metric based on integration over the tail of the RT distribution, exhibits good psychometric properties12,28 and has been used as primary outcome measure in much of the PVT literature, although other outcome measures such as mean RT have also commonly been used. The PVT shows a time-on-task effect—that is, a performance decrement From the Human Sleep and Cognition Laboratory, Sleep and Performance Research Center, Washington State University, Spokane, WA. This manuscript was received for review in September 2014. It was accepted for publication in February 2015. Address correspondence to: Kimberly A. Honn, Ph.D., Human Sleep and Cognition Laboratory, Sleep and Performance Research Center, Washington State University Spokane, P.O. Box 1495, Spokane, WA 99210; [email protected]. Reprint & Copyright © by the Aerospace Medical Association, Alexandria, VA. DOI: 10.3357/AMHP.4165.2015
May 2015
PORTABLE PVT VALIDATION —Honn et al.
with increasing RTs and increased variability of RTs over the study. During each of the 7 d prior to admission into the study, duration of a task—which is amplified by prior sleep loss.9,11,14 subjects filled out a sleep/wake diary, called in their bedtimes As such, the sensitivity of the PVT to fatigue is a function of the and wake times, and wore a wrist actigraph 24 h/d to estimate task duration. Furthermore, its sensitivity has been shown to sleep/wake patterns. Subjects’ average sleep duration (6 SD) as depend on the maximum intertrial interval.4,6,23 estimated by actigraphy was 8.0 (6 0.6) h/d. Subjects’ average Owing to extensive use of the PVT in research studies12,17 wake-up time (6 SD) as estimated by actigraphy was 07:34 (6 and systematic investigation of its properties and underlying 0.9 h). Subjects were financially compensated for their time. All brain mechanisms,4,7,24 the task has become a standard mea- subjects gave written, informed consent, and the study was sure of the effects of fatigue on reaction times during sustained approved by the Institutional Review Board of Washington vigilance performance both in the laboratory and in the field. State University. Nonetheless, deployment of the standard 10-min PVT in field environments has been hampered by the time requirement Procedure involved with repeated administration of the task20 and by limSubjects were studied inside the Human Sleep and Cognition ited availability of portable test platforms on which the task is Laboratory of the Sleep and Performance Research Center at implemented.22 Here we examine a version of the PVT devel- Washington State University Spokane. They participated in oped to remove this impediment. It is 5 min in duration and groups of three or four and were each assigned their own room implemented on a portable, touch-screen personal digital assisfor performance testing and for baseline and recovery sleep. tant (PDA) device. We conducted a laboratory study to validate They reported to the laboratory on Friday evening at 21:00 and this 5-min PDA PVT against the standard 10-min PVT across at 22:00 they went to bed for a baseline sleep period with scheda 38-h period of total sleep deprivation (TSD). uled awakening at 08:00 the next day. They then underwent a Delivered by Ingenta to: Purdue 38-h University TSD period.Libraries On Sunday evening at 22:00, subjects went to IP: 79.110.17.193 On: Wed,bed 03 Aug 02:31:15 for a 2016 recovery sleep period with scheduled awakening at Copyright: Aerospace Medical Association METHODS 08:00 the next day. They were discharged on Monday morning at 09:00, after having spent 60 h inside the laboratory. Following admission to the laboratory on Friday evening, Subjects There were 14 healthy subjects (50% women) who completed subjects were briefed on the study and then trained on the study test procedures. Under supervision by a research assistant in the study. Subjects’ ages ranged from 22 to 40 (average 29.9). Two subjects were left-handed. One subject (a 23-yr-old woman) their assigned rooms, subjects practiced the 10-min laptop PVT and then the 5-min PDA PVT, and filled out a paper-and-pencil developed symptoms of a pre-existing allergy during the study. In discussion with the physician of record, she completed the version of the Karolinska Sleepiness Scale (KSS).2 From Satstudy, but her data were not used for analysis. The final sample urday morning at 09:00 until Sunday evening at 20:00, a PVT size for the study is therefore N 5 13. was administered every hour, alternating between the 10-min Subjects were screened by telephone interview and during laptop PVT and the 5-min PDA PVT. Of the sample of 13 subtwo in-laboratory screening sessions to ensure they met eligi- jects, 7 started this sequence of alternating PVT versions with bility criteria. Subjects were physically and psychologically the laptop PVT, and 6 started with the PDA PVT. After each healthy and free of drugs and alcohol, as assessed with physical PVT, subjects filled out the KSS. Thus, during the 38-h TSD examination by the physician of record, blood chemistry, uri- period, a total of 18 laptop PVTs, 18 PDA PVTs, and 36 KSSs nalysis, breathalyzer test, history, and questionnaires. Subjects were administered. On Monday morning before discharge, the had no current psychiatric illnesses and no clinically relevant laptop PVT and the KSS were administered one more time to history of psychiatric illnesses, had no clinically relevant history verify that subjects had recovered. of brain injury, reported no learning disability, and had not experienced any adverse reaction to sleep deprivation. Female Equipment subjects were not pregnant. Subjects were not vision or hearing Subjects wore a wrist actigraph throughout the study to verify impaired (unless corrected to normal). They were not current scheduled sleep/wake patterns and they were monitored by smokers, had no current medical or drug treatment (excluding trained research assistants at all times. When not scheduled to oral contraceptives), had no history of drug or alcohol abuse in sleep and when not engaged in performance testing, subjects the past year, and had no history of methamphetamine abuse. undertook nonvigorous activities inside the laboratory (e.g., Urine drug test and breathalyzer test were repeated upon watching DVDs, playing board games) and interacted with admission into the laboratory to verify that subjects remained research assistants who made sure that subjects did not fall free of drugs and alcohol. asleep during TSD. Subjects were allowed to take a shower on Subjects reported to be good sleepers, with estimated sleep Sunday evening just before going to bed. Meals were served durations between 6 and 10 h per day and regular wake-up every 4 h during the TSD period and on Monday morning times between 06:00 and 09:00. They had no sleep or circadian before discharge. Caffeine was not permitted during the study. disorders as assessed by questionnaires and history. They had Subjects performed the laptop and PDA PVTs at a desk in not traveled across time zones within a month of entering the their assigned room. Both PVT test devices were located on the study and reported no shift work within 3 mo of entering the desk, arranged depending on the handedness of the subject so AEROSPACE MEDICINE AND HUMAN PERFORMANCE Vol. 86, No. 5
May 2015
429
PORTABLE PVT VALIDATION—Honn et al.
that the PDA was on the side of the subject’s dominant hand. During laptop testing, subjects were required to respond to a Both the laptop and the PDA were angled (in the horizontal stimulus in the form of a yellow, rolling millisecond counter, plane) toward the subject, who sat centered in front of the desk which appeared in a red box (3 cm wide and 1 cm tall) on the and swiveled the chair seat toward each test device. Each sub- screen (black background) at random ITIs. The minimum and ject’s chair was adjusted in height so that the subject could commaximum ITIs were set to 2 and 10 s, respectively. Subjects fortably operate the PVT test devices while sitting straight with were instructed to respond to the stimulus as fast as they could his or her back against the chair, feet flat on the floor. Subjects’ by pressing the space bar with their dominant hand while trying chairs were on gliders (not casters), and subjects were asked not to avoid false starts. A response froze the millisecond counter to move their chair once it was properly positioned. They were for 1 s to display the RT, after which the counter disappeared. asked to maintain a stable posture, not to lean their head on When a subject responded in the absence of a stimulus, “False their hand, and not to fidget during testing. Cameras were Start” appeared on the screen for 1 s and the next ITI began. If aimed at the subjects’ faces and at the overall set-up in the room a subject failed to respond in 30 s, a beep sounded and “Time to allow monitoring of compliance with test procedures from Out” showed on the screen for 1 s, after which the next ITI the laboratory’s observation room. began. The laptop PVT duration was set to 10 min. At the end The version of the PVT developed for the touch-screen PDA of a test bout, subjects did not receive any feedback about their (Boeing Company, Seattle, WA) was implemented on third- test performance. generation, 32-GB Apple iPods (Apple Inc., Cupertino, CA) with model number A1318 and running operating system Statistical Analyses version 5.1.1. The PDAs were each in a leather folio case with a PVT data analyses focused on the 10-min laptop PVT and ClearScreen overlay covering the screen (Belkin, Los Angeles, 5-min PDA PVT data from the 38-h TSD period. PVT numDelivered bystand, Ingenta to: Purdue CA). Each case was attached with Velcro to a rubber which ber ofUniversity lapses (RTsLibraries longer than 500 ms), mean RT, and false IP: 79.110.17.193 On: Wed, 03 Aug 02:31:15 was attached with Velcro to the desk. This kept the PDA steady starts were 2016 analyzed. As the two different PVT versions were Copyright: Aerospace Medical Association on the desk at approximately a 45° vertical angle. The PDA cases administered in alternating fashion, data were pooled over were closed at all times except during PDA PVT testing. 2-h blocks. Thus, there were 18 test times (2-h blocks) during During PDA testing, subjects were required to respond to a the 38-h TSD period across which to compare the laptop PVT stimulus in the form of a black bulls-eye (2 cm in diameter) and PDA PVT data. Primary statistical analyses involved which appeared on the screen (white background) at random mixed-effects analysis of variance (ANOVA)27 with test times intertrial intervals (ITIs). The minimum and maximum ITIs (every 2 h) as fixed effect, subjects as random effect on the were set to 2 and 10 s, respectively. Subjects were instructed to intercept, and order of PVT versions in each 2-h block (laptop respond to the stimulus as fast as they could by finger-tapping PVT first or PDA PVT first) as covariate. To compare the difthe touch screen with their dominant hand next to (not over) ferent PVT versions over equal amounts of time on task, these the stimulus, while trying to avoid false starts. When a subject analyses were repeated using only the first 5 min of the 10-min responded to the bulls-eye, the stimulus disappeared and the laptop PVT. For validation of the 5-min PDA PVT against the 10-min RT (in ms) was briefly shown. When a subject responded in the absence of a stimulus, “False Start” appeared on the screen laptop PVT, laptop PVT data were analyzed with PDA PVT briefly and the next ITI began. If a subject failed to respond in data on the same outcome measure as the only effect (covariate) and subjects as a random effect on the intercept. Valida10 s, the PDA showed “Time Expired” on the screen, produced tion results were expressed in terms of variance in the laptop two quick series of three beeps, and timed out. Following a PVT explained by the PDA PVT and in terms of correlation time-out, a “Continue” button appeared at the top of the screen which had to be pressed. An “Abort” button was present in the (square root of variance explained). For number of lapses, the top left corner of the screen during testing; subjects were asked local effect size of the effect of test times across the TSD period was assessed with Cohen’s f2,21 where 0.15 ⱕ f2 , 0.35 indinot to press this button and none did. The PDA PVT duration was set to 5 min. At the end of each test bout, the PDA screen cates medium effect size and f2 ⱖ 0.35 indicates large effect displayed the number of valid responses, number of false starts, size.8 number of time-outs, mean and standard deviation of RT, and To analyze the time-on-task effect, PVT responses for each minimum and maximum RT. test bout were arranged into 1-min response bins so that each Four individually calibrated Latitude E6420 laptops with 14″ laptop PVT test bout consisted of 10 bins and each PDA PVT displays (Dell Inc., Round Rock, TX), running the Windows 7 test bout consisted of 5 bins. The statistical analysis involved operating system (Microsoft Inc., Redmond, WA), were used in mixed-effects ANOVA of mean RT in the 1-min response bins, the study for administration of the standard PVT. The laptops with main effects of test time and response bin, a test time by ran PVT implementation version 2.0.5.9 (Pulsar Informatics response bin interaction, and a random effect on the intercept Inc., Philadelphia, PA). Each laptop was attached with Velcro to for subjects. The order of the PVT versions in each 2-h block the desk. The laptop display angle was adjusted to be perpen(laptop PVT first or PDA PVT first) was included as a covariate. dicular to the subject’s line of sight, with no glare on the screen. The main effect of response bin and the interaction of test time The laptops were kept in sleep mode at all times apart from lapby response bin capture the time-on-task effect, modulated by top PVT testing. homeostatic and circadian processes. 430
AEROSPACE MEDICINE AND HUMAN PERFORMANCE Vol. 86, No. 5
May 2015
PORTABLE PVT VALIDATION —Honn et al.
To examine any systematic differences in RT distributions between the PDA PVT and the laptop PVT, we compared the aggregated daytime (09:00–20:00) RT distributions on the two devices during baseline and after sleep deprivation. To independently verify the effectiveness of the TSD intervention and the equivalence of the two orders of task administration, the KSS data were analyzed as well. KSS sleepiness scores were analyzed using mixed-effects ANOVA with test times (every hour) as fixed effect, subjects as random effect on the intercept, and order of PVT versions in each 2-h block as covariate. Due to a technical failure, one PDA PVT was administered late to one of the subjects (at 02:21 instead of 02:00). The data from this test bout were excluded from analysis, resulting in a single missing data point, to which our statistical analyses were robust.15
RESULTS For PVT number of lapses, there were significant effects of test Ingenta to: Purdue University Libraries time for the 10-min laptop PVT [FDelivered (17,204) 5 by 7.52, P , 0.001] IP: 79.110.17.193 On: Wed, 03 Aug 2016 02:31:15 and for the 5-min PDA PVT [F(17,203) 5 4.01, P , 0.001] (see Copyright: Aerospace Medical Association Fig. 1, top panel). This indicates that, on both PVT platforms, performance changed dynamically across the 38 h of total sleep deprivation. When the analysis was restricted to the first 5 min of the 10-min laptop PVT, the effect of test time persisted [F(17,204) 5 3.72, P , 0.001]. For both PVT versions, there was no significant effect of the order of task administration on the number of lapses [laptop PVT: F(1,204) 5 2.35, P 5 0.13; PDA PVT: F(1,203) 5 1.99, P 5 0.16]. For laptop PVT lapses, the local effect size of the effect of test times across the TSD period was large (f2 5 0.36). For PDA PVT lapses, the local effect size of the effect of test times was medium (f2 5 0.17). The local effect size of the effect of test Fig. 1. Means and standard errors for outcome measures on the 10-min laptop times for lapses in the first 5 min of the 10-min laptop PVT was PVT (gray) and the 5-min PDA PVT (black) across 38 h of TSD in the laboratory. Plotted values are shown as centered in each 2-h block containing one laptop also medium (f2 5 0.20). As expected because of the difference in task duration PVT and one PDA PVT, and control for order of task administration. Top panel: number of lapses; middle panel: mean RT; bottom panel: number of false starts. between the laptop PVT (10 min) and the PDA PVT (5 min), the number of lapses observed during TSD was higher on the laptop PVT than on the PDA PVT. Nonetheless, number of Baseline mean RTs (observed during approximately the first lapses on the PDA PVT explained 49.6% of the variance in 12 h of the TSD period) were slower on the PDA PVT than on the laptop PVT. In addition, the range of change across the 38 h number of lapses on the laptop PVT [F(1,219) 5 170.3, P , 0.001]. That is, within and between subjects the correlation of TSD was smaller for the PDA PVT than for the laptop PVT. Nonetheless, mean RT on the PDA PVT explained 32.4% of the between the two PVT versions for number of lapses was 0.704. variance in mean RT on the laptop PVT [F(1,219) 5 76.27, P , For PVT mean RT, there were significant effects of test time 0.001]. That is, within and between subjects, the correlation for the laptop PVT [F(17,204) 5 5.97, P , 0.001] and for the between the two PVT versions for mean RT was 0.569. PDA PVT [F(17,203) 5 4.80, P , 0.001], indicating that averFor PVT number of false starts, there was a significant effect age reaction times changed dynamically across the 38 h of total sleep deprivation (see Fig. 1, middle panel). When the analysis of test time for the laptop PVT [F(17,204) 5 3.82, P , 0.001], indicating that errors of commission depended on when the was restricted to the first 5 min of the 10-min laptop PVT, the effect of mean RT across test times persisted [F(17,204) 5 5.92, test was taken across the TSD period, but not for the PDA PVT P , 0.001]. For both PVT versions, there was no significant [F(17,203) 5 1.47, P 5 0.11] (see Fig. 1, bottom panel). When effect of the order of task administration on the mean RTs, the analysis was restricted to the first 5 min of the 10-min lapalthough there was a trend for such an effect for the laptop PVT top PVT, the effect of false starts across test times persisted [laptop PVT: F(1,204) 5 2.91, P 5 0.089; PDA PVT: F(1,203) 5 [F(17,204) 5 1.85, P 5 0.024]. For both PVT versions, there was no significant effect of the order of task administration on 2.38, P 5 0.12]. AEROSPACE MEDICINE AND HUMAN PERFORMANCE Vol. 86, No. 5
May 2015
431
PORTABLE PVT VALIDATION—Honn et al.
For subjective sleepiness scores on the KSS as obtained durthe data [laptop PVT: F(1,204) 5 2.15, P 5 0.14; PDA PVT: ing the TSD period, there was a significant effect of test time F(1,203) 5 0.18, P 5 0.67]. The average number of false starts on the laptop PVT paral- [F(35,420) 5 39.62, P , 0.001] (see Fig. 4). The average sleepileled the average number of lapses (Fig. 1 top panel), as has ness scores on the KSS paralleled the average number of lapses on either version of the PVT (Fig. 1 top panel). There was no been documented in the literature.11 False starts on the PDA PVT did not vary as systematically over time and explained significant effect of the order of administration of the two different PVT versions [F(1,420) 5 0.01, P 5 0.93]. only 5.1% of the variance in number of false starts on the laptop PVT [a statistical trend: F(1,219) 5 3.28, P 5 0.071]. Within and between subjects, the correlation between the two PVT versions for number of false starts was 0.225. DISCUSSION Focusing on the time-on-task effect, for RT on the laptop PVT there was a significant effect of response bin [F(9,2148) 5 In this laboratory study with 38 h of TSD, the PDA PVT perfor7.80, P , 0.001] and a significant effect of test time [F(17,2148) mance testing functionality and results were comparable to the 5 21.45, P , 0.001]. Likewise, for mean RT on the PDA PVT, laptop PVT, and consistent with the published PVT literathere was a significant effect of response bin [F(4,1063) 5 4.21, ture.11,17,26 The PDA PVT lapses and mean RTs tracked the P 5 0.002] and a significant effect of test time [F(17,1063) 5 well-established interaction of homeostatic and circadian influ13.46, P , 0.001]. On both PVT devices, there was neither a ences on fatigue across hours of TSD [albeit with smaller significant test time by response bin interaction nor an effect of dynamic range than the laptop PVT (Fig. 1)]. These PVT outorder of task administration (see Fig. 2). When the analysis was come measures also paralleled changes in subjective sleepiness restricted to the first 5 min of the 10-min laptop PVT, the effects as assessed with the KSS (Fig. 4), as is typically found under 1 by Ingenta to: Purdue University of response bin [F(4,1068) 5 3.55, PDelivered 5 0.007] and of test time conditions of TSD.Libraries IP: 79.110.17.193 On: Wed, 03 Aug 2016 02:31:15 [F(17,1068) 5 13.08, P , 0.001] persisted. As expected, both the 10-min laptop PVT and the 5-min Copyright: Aerospace Medical Association To examine systematic differences in PVT RT distributions PDA PVT exhibited a time-on-task effect with worsening between the two devices, Fig. 3 shows the aggregated daytime performance (increasing RTs) over the course of each PVT RT distributions during baseline and after sleep deprivation. bout. Fig. 2 shows that the time-on-task effect was amplified Baseline RTs were slower overall on the PDA than on the laptop during the second day of the sleep deprivation period. This (PDA mean 5 326 ms; laptop mean 5 261 ms). Furthermore, reflects homeostatic and circadian modulation of the timebaseline RTs were somewhat more variable on the PDA than on on-task effect.11,25,30 Nonetheless, neither PVT version had a the laptop (PDA SD 5 76 ms; laptop SD 5 64 ms). However, statistically significant interaction between test time and bin, the change in the RT distribution due to sleep deprivation was probably because of the intrinsically low statistical power of more substantial on the laptop than on the PDA (laptop change interactions.31 in mean 5 134 ms; PDA change in mean 5 76 ms). The change Due to the shorter task duration of the PDA PVT, the magin spread of the RT distribution due to sleep deprivation was nitude of the time-on-task effect was smaller on the PDA PVT also more substantial on the laptop than on the PDA (laptop than on the laptop PVT (Fig. 2). The shorter task duration change in SD 5 466 ms; PDA change in SD 5 238 ms). of the PDA PVT also accommodated fewer stimuli, which reduced the signal-to-noise ratio. As a consequence of both effects, there were fewer lapses on the PDA PVT and the effect size for this outcome measure was smaller than on the laptop PVT. Premature responses, or false starts, often covary with lapses on the PVT and provide an indication of subjects’ effort to perform.11 On the PDA PVT, however, false starts did not parallel the temporal pattern of lapses, nor did they mirror false starts on the laptop PVT. This may be due to technical issues related to the PDA devices and/ or subjects’ steadiness of hand. Fig. 2. Means and standard errors for reaction times in 1-min bins on the 10-min laptop PVT (gray) and the 5-min PDA PVT On the PDA, a mere muscle (black) across 38 h of TSD in the laboratory. Plotted values are shown as beginning on clock times centered in each 2-h block twitch or accidental bump of the containing one laptop PVT and one PDA PVT, and are not to scale on the clock time axis for the timing of the 1-min bins. PDA touch screen may result in 432
AEROSPACE MEDICINE AND HUMAN PERFORMANCE Vol. 86, No. 5
May 2015
PORTABLE PVT VALIDATION —Honn et al.
To what extent the findings of our controlled laboratory TSD study generalize to use of the PDA PVT in field settings, to other kinds of fatiguing conditions, and to other populations cannot be readily inferred from the present data. For instance, administration of the PDA PVT with the device hand-held instead of mounted on a desk may increase RT variability and potentially reduce sensitivity to fatigue. Even so, the availability of a brief PVT implemented on a Fig. 3. Distribution of response times in 10-ms bins on the 10-min laptop PVT (gray) and 5-min PDA PVT (black) during portable, hand-held device fathe baseline day (solid lines) and sleep deprivation day (dashed lines). The insert shows the total number of RTs . 500 ms cilitates fatigue measurement in on the 10-min laptop PVT (gray) and 5-min PDA PVT (black) across the baseline day (solid bars) and sleep deprivation operational settings where time day (hatched bars). available for performance testing is limited and stationary test stabylaptop Ingenta to: Purdue Libraries a premature response. This is not anDelivered issue on the because tionsUniversity are not practical, such as resource extraction and transIP: 79.110.17.193 Wed,portation. 03 Aug 2016 more effortful motor action is required to press down theOn: space Other02:31:15 short-duration, hand-held PVTs have been Copyright: Aerospace Medical Association bar to respond. used successfully in published field studies, including studies Baseline RTs were generally slower on the PDA PVT than on with mining operators,13 pilots,18 and truck drivers.29 While the laptop PVT (Fig. 3). Baseline RTs were also more variable more research is needed, the 5-min PVT implemented on the on the PDA than on the laptop, which may be related to the touch-screen PDA would be expected to measure fatigue use of a touch screen on the PDA as opposed to a space bar on effectively in the field as well. In conclusion, the present study the laptop for responding. However, after sleep deprivation the found the 5-min PDA PVT to be a valid fatigue measurement changes in the RT distributions were more substantial on the tool, compared to a standard 10-min laptop PVT, in a 38-h laptop than on the PDA, both in terms of the mean and in terms sleep deprivation study in healthy young adults. of the SD. That is, in agreement with earlier observations,9,11,19 sleep deprivation caused skewing of the whole RT distribution to the right on both devices, but the degree of skewing was ACKNOWLEDGMENTS greater on the laptop than on the PDA. This was related to the 30 interaction of sleep deprivation with the time-on-task effect, which had much greater effect on the 10-min laptop PVT than We are grateful to the staff and research assistants of the Sleep and Performance Research Center at Washington State University Spokane for their help conon the 5-min PDA PVT. Even so, both versions of the PVT cap- ducting the study, and to Hans Van Dongen for guidance on research design tured the essence of the RT distribution changes that and execution and statistical analysis. We would like to thank Dr. Matthew characterize the effects of sleep deprivation on psychomotor Layton, the physician of record for the study. This study was supported by the Boeing Company. vigilance performance.5,9,19 Authors and affiliation: Kimberly A. Honn, Ph.D., Samantha M. Riedy, B.S., and Devon A. Grant, M.S., Human Sleep and Cognition Laboratory, Sleep and Performance Research Center, Washington State University, Spokane, WA.
REFERENCES
Fig. 4. Means and standard errors for KSS sleepiness scores reported hourly across 38 h of TSD in the laboratory. The KSS scale ranges from 1 (“extremely alert”) to 9 (“extremely sleepy – fighting sleep”). Plotted values control for order of administration of the two PVT versions.
1. Åkerstedt T, Anund A, Axelsson J, Kecklund G. Subjective sleepiness is a sensitive indicator of insufficient sleep and impaired waking function. J Sleep Res. 2014; 23(3):242–54. 2. Åkerstedt T, Gillberg M. Subjective and objective sleepiness in the active individual. Int J Neurosci. 1990; 52(1-2):29–37. 3. Balkin TJ, Bliese PD, Belenky G, Sing H, Thorne DR, et al. Comparative utility of instruments for monitoring sleepiness-related performance decrements in the operational environment. J Sleep Res. 2004; 13(3):219–227. 4. Basner M, Dinges DF. Maximizing sensitivity of the psychomotor vigilance test (PVT) to sleep loss. Sleep. 2011; 34(5):581–591.
AEROSPACE MEDICINE AND HUMAN PERFORMANCE Vol. 86, No. 5
May 2015
433
PORTABLE PVT VALIDATION—Honn et al.
5. Basner M, Mollicone D, Dinges DF. Validity and sensitivity of a brief 19. Ratcliff R, Van Dongen HPA. Diffusion model for one-choice reactionPsychomotor Vigilance Test (PVT-B) to total and partial sleep deprivatime tasks and the cognitive effects of sleep deprivation. Proc Natl Acad tion. Acta Astronaut. 2011; 69(11-12):949–959. Sci U S A. 2011; 108(27):11285–11290. 6. Basner M, Rubinstein J. Fitness for duty: a 3-minute version of the 20. Roach GD, Dawson D, Lamond N. Can a shorter psychomotor vigilance Psychomotor Vigilance Test predicts fatigue related declines in luggage task be used as a reasonable substitute for the ten-minute psychomotor screening performance. J Occup Environ Med. 2011; 53(10):1146–1154. vigilance task? Chronobiol Int. 2006; 23(6):1379–1387. 7. Chee MWL, Chuah LYM, Venkatraman V, Chan WY, Philip P, Dinges DF. 21. Selya AS, Rose JS, Dierker LC, Hedeker D, Mermelstein RJ. A practical Functional imaging of working memory following normal sleep and after guide to calculating Cohen’s f2, a measure of local effect size, from PROC 24 and 35 h of sleep deprivation: correlations of fronto-parietal activation MIXED. Front Psychol. 2012; 3:111. with performance. Neuroimage. 2006; 31(1):419–428. 22. Thorne DR, Johnson DE, Redmond DP, Sing HC, Belenky G, Shapiro 8. Cohen JE. Statistical power analysis for the behavioral sciences, 2nd rev. JM. The Walter Reed palm-held psychomotor vigilance test. Behav Res ed. Hillsdale (NJ): Lawrence Erlbaum Associates; 1988:410–414. Methods. 2005; 37(1):111–118. 9. Dinges DF, Kribbs NB. Performing while sleepy: effects of experimentally23. Tucker AM, Basner RC, Stern Y, Rakitin BC. The variable responseinduced sleepiness. In: Monk TH, editor. Sleep, sleepiness, and perforstimulus interval effect and sleep deprivation: an unexplored aspect of mance. New York: Wiley; 1991:97–128. psychomotor vigilance task performance. Sleep. 2009; 32(10):1393–1395. 10. Dinges DF, Powell JW. Microcomputer analyses of performance on a 24. Van Dongen HP, Belenky G, Krueger JM. A local, bottom-up perspective portable, simple visual RT task during sustained operations. Behav Res on sleep deprivation and neurobehavioral performance. Curr Top Med Methods Instrum Comput. 1985; 17(6):652–655. Chem. 2011; 11(19):2414–2422. 11. Doran SM, Van Dongen HPA, Dinges DF. Sustained attention performance 25. Van Dongen HPA, Belenky G, Krueger JM. Investigating the temporal during sleep deprivation: evidence of state instability. Arch Ital Biol. 2001; dynamics and underlying mechanisms of cognitive fatigue. In: Ackerman 139(3):253–267. PL, editor. Cognitive fatigue: multidisciplinary perspectives on current 12. Dorrian J, Rogers NL, Dinges DF. Psychomotor vigilance performance: research and future applications. Washington (DC): American Psychoneurocognitive assay sensitive to sleep loss. In: Kushida CA, editor. Sleep logical Association; 2011:127–147. deprivation: clinical issues, pharmacology, and sleep loss effects. New 26. Van Dongen HPA, Dinges DF. Sleep, circadian rhythms, and psychomotor Delivered by Ingenta to: Purduevigilance University Libraries York: Marcel Dekker; 2005:39–70. . Clin Sports Med. 2005; 24(2):237–249. IP:GD, 79.110.17.193 On:onWed,27.03Van Aug 2016 02:31:15 13. Ferguson SA, Paech GM, Dorrian J, Roach Jay SM. Performance Dongen HPA, Maislin G, Dinges DF. Dealing with inter-individual Copyright: Aerospace Medical Association a simple response time task: is sleep or work more important for miners? differences in the temporal dynamics of fatigue and performance: imAppl Ergon. 2011; 42(2):210–213. portance and techniques. Aviat Space Environ Med. 2004; 75(3, Suppl.): 14. Kribbs NB, Dinges DF. Vigilance decrement and sleepiness. In: Harsh A147–A154. JR, Ogilvie RD, editors. Sleep onset mechanisms. Washington (DC): 28. Van Dongen HPA, Maislin G, Mullington JM, Dinges DF. The cumulative American Psychological Association; 1994:113–125. cost of additional wakefulness: dose-response effects on neurobehavioral 15. Krueger C, Tian L. A comparison of the general linear mixed model and functions and sleep physiology from chronic sleep restriction and total repeated measures ANOVA using a dataset with multiple missing data sleep deprivation. Sleep. 2003; 26(2):117–126. points. Biol Res Nurs. 2004; 6(2):151–157. 29. Van Dongen HPA, Mollicone DJ. Field study on the efficacy of the new 16. Lim J, Dinges DF. A meta-analysis of the impact of short-term sleep restart provision for hours of service. Washington (DC): Federal Motor deprivation on cognitive variables. Psychol Bull. 2010; 136(3):375–389. Carrier Safety Administration; 2013. Report No: RRR-13-058. 17. Lim J, Dinges DF. Sleep deprivation and vigilant attention. Ann N Y Acad 30. Wesensten NJ, Belenky G, Thorne DR, Kautz MA, Balkin TJ. Modafinil Sci. 2008; 1129:305–322. vs. caffeine: effects on fatigue during sleep deprivation. Aviat Space 18. Petrilli RM, Roach GD, Dawson D, Lamond N. The sleep, subjective Environ Med. 2004; 75(6):520–525. fatigue, and sustained attention of commercial airline pilots during an 31. Winer BJ. Statistical principles in experimental design, 2nd ed. New York: international pattern. Chronobiol Int. 2006; 23(6):1357–1362. McGraw Hill; 1971:378–384.
434
AEROSPACE MEDICINE AND HUMAN PERFORMANCE Vol. 86, No. 5
May 2015
Validation of a Portable, Touch-Screen Psychomotor Vigilance Test Kimberly A. Honn; Samantha M. Riedy; Devon A. Grant
INTRODUCTION:
METHODS:
RESULTS:
DISCUSSION:
KEYWORDS:
The Psychomotor Vigilance Test (PVT) measures effects of fatigue from sleep loss and circadian misalignment on sustained vigilance performance. To promote PVT use in field environments, a 5-min PVT version has been implemented on a personal digital Delivered assistant (PDA) a touch TheUniversity present laboratory study was conducted to validate this by with Ingenta to:screen. Purdue Libraries PVT against a standard laptop PVT across 38 h of03 total sleep deprivation (TSD). IP:10-min 79.110.17.193 On: Wed, Aug 2016 02:31:15
Aerospace Association Following a baseline sleepCopyright: night, subjects underwent Medical 38 h of TSD, during which they performed the PVT every hour, alternating between the two test platforms. The study concluded with a night of recovery sleep. The primary outcome was the number of PVT lapses (reaction times . 500 ms). Both PVT platforms showed significant effects for the number of lapses across TSD test times involving an increase with time awake modulated by circadian rhythm. Laptop PVT lapses across test times exhibited a large effect size (f2 5 0.36), whereas PDA PVT lapses exhibited a medium effect size (f2 5 0.17). The laptop PVT showed a significant effect for the number of false starts during TSD similar to the temporal profile of lapses, while the PDA PVT had false starts throughout the TSD period. The 5-min PDA PVT provided performance testing functionality and results comparable to the 10-min laptop PVT. The number of PDA PVT lapses tracked fatigue similarly to the laptop PVT lapses, albeit with smaller average ranges and effect sizes. sleep deprivation, fatigue, neurobehavioral performance, PVT, lapses of attention. Honn KA, Riedy SM, Grant DA. Validation of a portable, touch-screen psychomotor vigilance test. Aerosp Med Hum Perform. 2015; 86(5):428–434.
T
he psychomotor vigilance test,10,17 commonly referred to by the acronym PVT, is a computerized serial reaction time (RT) test requiring subjects to respond, by means of button presses, to stimuli appearing on a screen at random intervals. The PVT’s requirement for sustained attention (typically for 3, 5, 10, or 20 min per test bout) and its high stimulus density (on the order of about 10 stimuli per minute or more) make the test highly sensitive to fatigue (sleepiness) from sleep loss, circadian misalignment, and sleep disorders.3,16 The PVT records RTs, from which a variety of outcome measures can be computed, such as mean or median RT, 10% fastest and 10% slowest RTs (and transformations thereof), number of lapses (conventionally defined as RTs longer than a 500-ms threshold), and number of false starts (errors of commission). These outcome measures capture different aspects of the underlying RT distribution, which displays a long right-hand tail. Fatigue causes progressively more RTs to end up in the right-hand tail.9,11 This shift can be adequately described by a one-parameter shift in one of the RT
428
AEROSPACE MEDICINE AND HUMAN PERFORMANCE Vol. 86, No. 5
distribution parameters.19 As such, in the context of measuring fatigue, different PVT outcome measures capture essentially the same phenomenon, and selection of a primary outcome measure hinges primarily on signal-to-noise ratio and other practical considerations. The number of lapses, a metric based on integration over the tail of the RT distribution, exhibits good psychometric properties12,28 and has been used as primary outcome measure in much of the PVT literature, although other outcome measures such as mean RT have also commonly been used. The PVT shows a time-on-task effect—that is, a performance decrement From the Human Sleep and Cognition Laboratory, Sleep and Performance Research Center, Washington State University, Spokane, WA. This manuscript was received for review in September 2014. It was accepted for publication in February 2015. Address correspondence to: Kimberly A. Honn, Ph.D., Human Sleep and Cognition Laboratory, Sleep and Performance Research Center, Washington State University Spokane, P.O. Box 1495, Spokane, WA 99210; [email protected]. Reprint & Copyright © by the Aerospace Medical Association, Alexandria, VA. DOI: 10.3357/AMHP.4165.2015
May 2015
PORTABLE PVT VALIDATION —Honn et al.
with increasing RTs and increased variability of RTs over the study. During each of the 7 d prior to admission into the study, duration of a task—which is amplified by prior sleep loss.9,11,14 subjects filled out a sleep/wake diary, called in their bedtimes As such, the sensitivity of the PVT to fatigue is a function of the and wake times, and wore a wrist actigraph 24 h/d to estimate task duration. Furthermore, its sensitivity has been shown to sleep/wake patterns. Subjects’ average sleep duration (6 SD) as depend on the maximum intertrial interval.4,6,23 estimated by actigraphy was 8.0 (6 0.6) h/d. Subjects’ average Owing to extensive use of the PVT in research studies12,17 wake-up time (6 SD) as estimated by actigraphy was 07:34 (6 and systematic investigation of its properties and underlying 0.9 h). Subjects were financially compensated for their time. All brain mechanisms,4,7,24 the task has become a standard mea- subjects gave written, informed consent, and the study was sure of the effects of fatigue on reaction times during sustained approved by the Institutional Review Board of Washington vigilance performance both in the laboratory and in the field. State University. Nonetheless, deployment of the standard 10-min PVT in field environments has been hampered by the time requirement Procedure involved with repeated administration of the task20 and by limSubjects were studied inside the Human Sleep and Cognition ited availability of portable test platforms on which the task is Laboratory of the Sleep and Performance Research Center at implemented.22 Here we examine a version of the PVT devel- Washington State University Spokane. They participated in oped to remove this impediment. It is 5 min in duration and groups of three or four and were each assigned their own room implemented on a portable, touch-screen personal digital assisfor performance testing and for baseline and recovery sleep. tant (PDA) device. We conducted a laboratory study to validate They reported to the laboratory on Friday evening at 21:00 and this 5-min PDA PVT against the standard 10-min PVT across at 22:00 they went to bed for a baseline sleep period with scheda 38-h period of total sleep deprivation (TSD). uled awakening at 08:00 the next day. They then underwent a Delivered by Ingenta to: Purdue 38-h University TSD period.Libraries On Sunday evening at 22:00, subjects went to IP: 79.110.17.193 On: Wed,bed 03 Aug 02:31:15 for a 2016 recovery sleep period with scheduled awakening at Copyright: Aerospace Medical Association METHODS 08:00 the next day. They were discharged on Monday morning at 09:00, after having spent 60 h inside the laboratory. Following admission to the laboratory on Friday evening, Subjects There were 14 healthy subjects (50% women) who completed subjects were briefed on the study and then trained on the study test procedures. Under supervision by a research assistant in the study. Subjects’ ages ranged from 22 to 40 (average 29.9). Two subjects were left-handed. One subject (a 23-yr-old woman) their assigned rooms, subjects practiced the 10-min laptop PVT and then the 5-min PDA PVT, and filled out a paper-and-pencil developed symptoms of a pre-existing allergy during the study. In discussion with the physician of record, she completed the version of the Karolinska Sleepiness Scale (KSS).2 From Satstudy, but her data were not used for analysis. The final sample urday morning at 09:00 until Sunday evening at 20:00, a PVT size for the study is therefore N 5 13. was administered every hour, alternating between the 10-min Subjects were screened by telephone interview and during laptop PVT and the 5-min PDA PVT. Of the sample of 13 subtwo in-laboratory screening sessions to ensure they met eligi- jects, 7 started this sequence of alternating PVT versions with bility criteria. Subjects were physically and psychologically the laptop PVT, and 6 started with the PDA PVT. After each healthy and free of drugs and alcohol, as assessed with physical PVT, subjects filled out the KSS. Thus, during the 38-h TSD examination by the physician of record, blood chemistry, uri- period, a total of 18 laptop PVTs, 18 PDA PVTs, and 36 KSSs nalysis, breathalyzer test, history, and questionnaires. Subjects were administered. On Monday morning before discharge, the had no current psychiatric illnesses and no clinically relevant laptop PVT and the KSS were administered one more time to history of psychiatric illnesses, had no clinically relevant history verify that subjects had recovered. of brain injury, reported no learning disability, and had not experienced any adverse reaction to sleep deprivation. Female Equipment subjects were not pregnant. Subjects were not vision or hearing Subjects wore a wrist actigraph throughout the study to verify impaired (unless corrected to normal). They were not current scheduled sleep/wake patterns and they were monitored by smokers, had no current medical or drug treatment (excluding trained research assistants at all times. When not scheduled to oral contraceptives), had no history of drug or alcohol abuse in sleep and when not engaged in performance testing, subjects the past year, and had no history of methamphetamine abuse. undertook nonvigorous activities inside the laboratory (e.g., Urine drug test and breathalyzer test were repeated upon watching DVDs, playing board games) and interacted with admission into the laboratory to verify that subjects remained research assistants who made sure that subjects did not fall free of drugs and alcohol. asleep during TSD. Subjects were allowed to take a shower on Subjects reported to be good sleepers, with estimated sleep Sunday evening just before going to bed. Meals were served durations between 6 and 10 h per day and regular wake-up every 4 h during the TSD period and on Monday morning times between 06:00 and 09:00. They had no sleep or circadian before discharge. Caffeine was not permitted during the study. disorders as assessed by questionnaires and history. They had Subjects performed the laptop and PDA PVTs at a desk in not traveled across time zones within a month of entering the their assigned room. Both PVT test devices were located on the study and reported no shift work within 3 mo of entering the desk, arranged depending on the handedness of the subject so AEROSPACE MEDICINE AND HUMAN PERFORMANCE Vol. 86, No. 5
May 2015
429
PORTABLE PVT VALIDATION—Honn et al.
that the PDA was on the side of the subject’s dominant hand. During laptop testing, subjects were required to respond to a Both the laptop and the PDA were angled (in the horizontal stimulus in the form of a yellow, rolling millisecond counter, plane) toward the subject, who sat centered in front of the desk which appeared in a red box (3 cm wide and 1 cm tall) on the and swiveled the chair seat toward each test device. Each sub- screen (black background) at random ITIs. The minimum and ject’s chair was adjusted in height so that the subject could commaximum ITIs were set to 2 and 10 s, respectively. Subjects fortably operate the PVT test devices while sitting straight with were instructed to respond to the stimulus as fast as they could his or her back against the chair, feet flat on the floor. Subjects’ by pressing the space bar with their dominant hand while trying chairs were on gliders (not casters), and subjects were asked not to avoid false starts. A response froze the millisecond counter to move their chair once it was properly positioned. They were for 1 s to display the RT, after which the counter disappeared. asked to maintain a stable posture, not to lean their head on When a subject responded in the absence of a stimulus, “False their hand, and not to fidget during testing. Cameras were Start” appeared on the screen for 1 s and the next ITI began. If aimed at the subjects’ faces and at the overall set-up in the room a subject failed to respond in 30 s, a beep sounded and “Time to allow monitoring of compliance with test procedures from Out” showed on the screen for 1 s, after which the next ITI the laboratory’s observation room. began. The laptop PVT duration was set to 10 min. At the end The version of the PVT developed for the touch-screen PDA of a test bout, subjects did not receive any feedback about their (Boeing Company, Seattle, WA) was implemented on third- test performance. generation, 32-GB Apple iPods (Apple Inc., Cupertino, CA) with model number A1318 and running operating system Statistical Analyses version 5.1.1. The PDAs were each in a leather folio case with a PVT data analyses focused on the 10-min laptop PVT and ClearScreen overlay covering the screen (Belkin, Los Angeles, 5-min PDA PVT data from the 38-h TSD period. PVT numDelivered bystand, Ingenta to: Purdue CA). Each case was attached with Velcro to a rubber which ber ofUniversity lapses (RTsLibraries longer than 500 ms), mean RT, and false IP: 79.110.17.193 On: Wed, 03 Aug 02:31:15 was attached with Velcro to the desk. This kept the PDA steady starts were 2016 analyzed. As the two different PVT versions were Copyright: Aerospace Medical Association on the desk at approximately a 45° vertical angle. The PDA cases administered in alternating fashion, data were pooled over were closed at all times except during PDA PVT testing. 2-h blocks. Thus, there were 18 test times (2-h blocks) during During PDA testing, subjects were required to respond to a the 38-h TSD period across which to compare the laptop PVT stimulus in the form of a black bulls-eye (2 cm in diameter) and PDA PVT data. Primary statistical analyses involved which appeared on the screen (white background) at random mixed-effects analysis of variance (ANOVA)27 with test times intertrial intervals (ITIs). The minimum and maximum ITIs (every 2 h) as fixed effect, subjects as random effect on the were set to 2 and 10 s, respectively. Subjects were instructed to intercept, and order of PVT versions in each 2-h block (laptop respond to the stimulus as fast as they could by finger-tapping PVT first or PDA PVT first) as covariate. To compare the difthe touch screen with their dominant hand next to (not over) ferent PVT versions over equal amounts of time on task, these the stimulus, while trying to avoid false starts. When a subject analyses were repeated using only the first 5 min of the 10-min responded to the bulls-eye, the stimulus disappeared and the laptop PVT. For validation of the 5-min PDA PVT against the 10-min RT (in ms) was briefly shown. When a subject responded in the absence of a stimulus, “False Start” appeared on the screen laptop PVT, laptop PVT data were analyzed with PDA PVT briefly and the next ITI began. If a subject failed to respond in data on the same outcome measure as the only effect (covariate) and subjects as a random effect on the intercept. Valida10 s, the PDA showed “Time Expired” on the screen, produced tion results were expressed in terms of variance in the laptop two quick series of three beeps, and timed out. Following a PVT explained by the PDA PVT and in terms of correlation time-out, a “Continue” button appeared at the top of the screen which had to be pressed. An “Abort” button was present in the (square root of variance explained). For number of lapses, the top left corner of the screen during testing; subjects were asked local effect size of the effect of test times across the TSD period was assessed with Cohen’s f2,21 where 0.15 ⱕ f2 , 0.35 indinot to press this button and none did. The PDA PVT duration was set to 5 min. At the end of each test bout, the PDA screen cates medium effect size and f2 ⱖ 0.35 indicates large effect displayed the number of valid responses, number of false starts, size.8 number of time-outs, mean and standard deviation of RT, and To analyze the time-on-task effect, PVT responses for each minimum and maximum RT. test bout were arranged into 1-min response bins so that each Four individually calibrated Latitude E6420 laptops with 14″ laptop PVT test bout consisted of 10 bins and each PDA PVT displays (Dell Inc., Round Rock, TX), running the Windows 7 test bout consisted of 5 bins. The statistical analysis involved operating system (Microsoft Inc., Redmond, WA), were used in mixed-effects ANOVA of mean RT in the 1-min response bins, the study for administration of the standard PVT. The laptops with main effects of test time and response bin, a test time by ran PVT implementation version 2.0.5.9 (Pulsar Informatics response bin interaction, and a random effect on the intercept Inc., Philadelphia, PA). Each laptop was attached with Velcro to for subjects. The order of the PVT versions in each 2-h block the desk. The laptop display angle was adjusted to be perpen(laptop PVT first or PDA PVT first) was included as a covariate. dicular to the subject’s line of sight, with no glare on the screen. The main effect of response bin and the interaction of test time The laptops were kept in sleep mode at all times apart from lapby response bin capture the time-on-task effect, modulated by top PVT testing. homeostatic and circadian processes. 430
AEROSPACE MEDICINE AND HUMAN PERFORMANCE Vol. 86, No. 5
May 2015
PORTABLE PVT VALIDATION —Honn et al.
To examine any systematic differences in RT distributions between the PDA PVT and the laptop PVT, we compared the aggregated daytime (09:00–20:00) RT distributions on the two devices during baseline and after sleep deprivation. To independently verify the effectiveness of the TSD intervention and the equivalence of the two orders of task administration, the KSS data were analyzed as well. KSS sleepiness scores were analyzed using mixed-effects ANOVA with test times (every hour) as fixed effect, subjects as random effect on the intercept, and order of PVT versions in each 2-h block as covariate. Due to a technical failure, one PDA PVT was administered late to one of the subjects (at 02:21 instead of 02:00). The data from this test bout were excluded from analysis, resulting in a single missing data point, to which our statistical analyses were robust.15
RESULTS For PVT number of lapses, there were significant effects of test Ingenta to: Purdue University Libraries time for the 10-min laptop PVT [FDelivered (17,204) 5 by 7.52, P , 0.001] IP: 79.110.17.193 On: Wed, 03 Aug 2016 02:31:15 and for the 5-min PDA PVT [F(17,203) 5 4.01, P , 0.001] (see Copyright: Aerospace Medical Association Fig. 1, top panel). This indicates that, on both PVT platforms, performance changed dynamically across the 38 h of total sleep deprivation. When the analysis was restricted to the first 5 min of the 10-min laptop PVT, the effect of test time persisted [F(17,204) 5 3.72, P , 0.001]. For both PVT versions, there was no significant effect of the order of task administration on the number of lapses [laptop PVT: F(1,204) 5 2.35, P 5 0.13; PDA PVT: F(1,203) 5 1.99, P 5 0.16]. For laptop PVT lapses, the local effect size of the effect of test times across the TSD period was large (f2 5 0.36). For PDA PVT lapses, the local effect size of the effect of test times was medium (f2 5 0.17). The local effect size of the effect of test Fig. 1. Means and standard errors for outcome measures on the 10-min laptop times for lapses in the first 5 min of the 10-min laptop PVT was PVT (gray) and the 5-min PDA PVT (black) across 38 h of TSD in the laboratory. Plotted values are shown as centered in each 2-h block containing one laptop also medium (f2 5 0.20). As expected because of the difference in task duration PVT and one PDA PVT, and control for order of task administration. Top panel: number of lapses; middle panel: mean RT; bottom panel: number of false starts. between the laptop PVT (10 min) and the PDA PVT (5 min), the number of lapses observed during TSD was higher on the laptop PVT than on the PDA PVT. Nonetheless, number of Baseline mean RTs (observed during approximately the first lapses on the PDA PVT explained 49.6% of the variance in 12 h of the TSD period) were slower on the PDA PVT than on the laptop PVT. In addition, the range of change across the 38 h number of lapses on the laptop PVT [F(1,219) 5 170.3, P , 0.001]. That is, within and between subjects the correlation of TSD was smaller for the PDA PVT than for the laptop PVT. Nonetheless, mean RT on the PDA PVT explained 32.4% of the between the two PVT versions for number of lapses was 0.704. variance in mean RT on the laptop PVT [F(1,219) 5 76.27, P , For PVT mean RT, there were significant effects of test time 0.001]. That is, within and between subjects, the correlation for the laptop PVT [F(17,204) 5 5.97, P , 0.001] and for the between the two PVT versions for mean RT was 0.569. PDA PVT [F(17,203) 5 4.80, P , 0.001], indicating that averFor PVT number of false starts, there was a significant effect age reaction times changed dynamically across the 38 h of total sleep deprivation (see Fig. 1, middle panel). When the analysis of test time for the laptop PVT [F(17,204) 5 3.82, P , 0.001], indicating that errors of commission depended on when the was restricted to the first 5 min of the 10-min laptop PVT, the effect of mean RT across test times persisted [F(17,204) 5 5.92, test was taken across the TSD period, but not for the PDA PVT P , 0.001]. For both PVT versions, there was no significant [F(17,203) 5 1.47, P 5 0.11] (see Fig. 1, bottom panel). When effect of the order of task administration on the mean RTs, the analysis was restricted to the first 5 min of the 10-min lapalthough there was a trend for such an effect for the laptop PVT top PVT, the effect of false starts across test times persisted [laptop PVT: F(1,204) 5 2.91, P 5 0.089; PDA PVT: F(1,203) 5 [F(17,204) 5 1.85, P 5 0.024]. For both PVT versions, there was no significant effect of the order of task administration on 2.38, P 5 0.12]. AEROSPACE MEDICINE AND HUMAN PERFORMANCE Vol. 86, No. 5
May 2015
431
PORTABLE PVT VALIDATION—Honn et al.
For subjective sleepiness scores on the KSS as obtained durthe data [laptop PVT: F(1,204) 5 2.15, P 5 0.14; PDA PVT: ing the TSD period, there was a significant effect of test time F(1,203) 5 0.18, P 5 0.67]. The average number of false starts on the laptop PVT paral- [F(35,420) 5 39.62, P , 0.001] (see Fig. 4). The average sleepileled the average number of lapses (Fig. 1 top panel), as has ness scores on the KSS paralleled the average number of lapses on either version of the PVT (Fig. 1 top panel). There was no been documented in the literature.11 False starts on the PDA PVT did not vary as systematically over time and explained significant effect of the order of administration of the two different PVT versions [F(1,420) 5 0.01, P 5 0.93]. only 5.1% of the variance in number of false starts on the laptop PVT [a statistical trend: F(1,219) 5 3.28, P 5 0.071]. Within and between subjects, the correlation between the two PVT versions for number of false starts was 0.225. DISCUSSION Focusing on the time-on-task effect, for RT on the laptop PVT there was a significant effect of response bin [F(9,2148) 5 In this laboratory study with 38 h of TSD, the PDA PVT perfor7.80, P , 0.001] and a significant effect of test time [F(17,2148) mance testing functionality and results were comparable to the 5 21.45, P , 0.001]. Likewise, for mean RT on the PDA PVT, laptop PVT, and consistent with the published PVT literathere was a significant effect of response bin [F(4,1063) 5 4.21, ture.11,17,26 The PDA PVT lapses and mean RTs tracked the P 5 0.002] and a significant effect of test time [F(17,1063) 5 well-established interaction of homeostatic and circadian influ13.46, P , 0.001]. On both PVT devices, there was neither a ences on fatigue across hours of TSD [albeit with smaller significant test time by response bin interaction nor an effect of dynamic range than the laptop PVT (Fig. 1)]. These PVT outorder of task administration (see Fig. 2). When the analysis was come measures also paralleled changes in subjective sleepiness restricted to the first 5 min of the 10-min laptop PVT, the effects as assessed with the KSS (Fig. 4), as is typically found under 1 by Ingenta to: Purdue University of response bin [F(4,1068) 5 3.55, PDelivered 5 0.007] and of test time conditions of TSD.Libraries IP: 79.110.17.193 On: Wed, 03 Aug 2016 02:31:15 [F(17,1068) 5 13.08, P , 0.001] persisted. As expected, both the 10-min laptop PVT and the 5-min Copyright: Aerospace Medical Association To examine systematic differences in PVT RT distributions PDA PVT exhibited a time-on-task effect with worsening between the two devices, Fig. 3 shows the aggregated daytime performance (increasing RTs) over the course of each PVT RT distributions during baseline and after sleep deprivation. bout. Fig. 2 shows that the time-on-task effect was amplified Baseline RTs were slower overall on the PDA than on the laptop during the second day of the sleep deprivation period. This (PDA mean 5 326 ms; laptop mean 5 261 ms). Furthermore, reflects homeostatic and circadian modulation of the timebaseline RTs were somewhat more variable on the PDA than on on-task effect.11,25,30 Nonetheless, neither PVT version had a the laptop (PDA SD 5 76 ms; laptop SD 5 64 ms). However, statistically significant interaction between test time and bin, the change in the RT distribution due to sleep deprivation was probably because of the intrinsically low statistical power of more substantial on the laptop than on the PDA (laptop change interactions.31 in mean 5 134 ms; PDA change in mean 5 76 ms). The change Due to the shorter task duration of the PDA PVT, the magin spread of the RT distribution due to sleep deprivation was nitude of the time-on-task effect was smaller on the PDA PVT also more substantial on the laptop than on the PDA (laptop than on the laptop PVT (Fig. 2). The shorter task duration change in SD 5 466 ms; PDA change in SD 5 238 ms). of the PDA PVT also accommodated fewer stimuli, which reduced the signal-to-noise ratio. As a consequence of both effects, there were fewer lapses on the PDA PVT and the effect size for this outcome measure was smaller than on the laptop PVT. Premature responses, or false starts, often covary with lapses on the PVT and provide an indication of subjects’ effort to perform.11 On the PDA PVT, however, false starts did not parallel the temporal pattern of lapses, nor did they mirror false starts on the laptop PVT. This may be due to technical issues related to the PDA devices and/ or subjects’ steadiness of hand. Fig. 2. Means and standard errors for reaction times in 1-min bins on the 10-min laptop PVT (gray) and the 5-min PDA PVT On the PDA, a mere muscle (black) across 38 h of TSD in the laboratory. Plotted values are shown as beginning on clock times centered in each 2-h block twitch or accidental bump of the containing one laptop PVT and one PDA PVT, and are not to scale on the clock time axis for the timing of the 1-min bins. PDA touch screen may result in 432
AEROSPACE MEDICINE AND HUMAN PERFORMANCE Vol. 86, No. 5
May 2015
PORTABLE PVT VALIDATION —Honn et al.
To what extent the findings of our controlled laboratory TSD study generalize to use of the PDA PVT in field settings, to other kinds of fatiguing conditions, and to other populations cannot be readily inferred from the present data. For instance, administration of the PDA PVT with the device hand-held instead of mounted on a desk may increase RT variability and potentially reduce sensitivity to fatigue. Even so, the availability of a brief PVT implemented on a Fig. 3. Distribution of response times in 10-ms bins on the 10-min laptop PVT (gray) and 5-min PDA PVT (black) during portable, hand-held device fathe baseline day (solid lines) and sleep deprivation day (dashed lines). The insert shows the total number of RTs . 500 ms cilitates fatigue measurement in on the 10-min laptop PVT (gray) and 5-min PDA PVT (black) across the baseline day (solid bars) and sleep deprivation operational settings where time day (hatched bars). available for performance testing is limited and stationary test stabylaptop Ingenta to: Purdue Libraries a premature response. This is not anDelivered issue on the because tionsUniversity are not practical, such as resource extraction and transIP: 79.110.17.193 Wed,portation. 03 Aug 2016 more effortful motor action is required to press down theOn: space Other02:31:15 short-duration, hand-held PVTs have been Copyright: Aerospace Medical Association bar to respond. used successfully in published field studies, including studies Baseline RTs were generally slower on the PDA PVT than on with mining operators,13 pilots,18 and truck drivers.29 While the laptop PVT (Fig. 3). Baseline RTs were also more variable more research is needed, the 5-min PVT implemented on the on the PDA than on the laptop, which may be related to the touch-screen PDA would be expected to measure fatigue use of a touch screen on the PDA as opposed to a space bar on effectively in the field as well. In conclusion, the present study the laptop for responding. However, after sleep deprivation the found the 5-min PDA PVT to be a valid fatigue measurement changes in the RT distributions were more substantial on the tool, compared to a standard 10-min laptop PVT, in a 38-h laptop than on the PDA, both in terms of the mean and in terms sleep deprivation study in healthy young adults. of the SD. That is, in agreement with earlier observations,9,11,19 sleep deprivation caused skewing of the whole RT distribution to the right on both devices, but the degree of skewing was ACKNOWLEDGMENTS greater on the laptop than on the PDA. This was related to the 30 interaction of sleep deprivation with the time-on-task effect, which had much greater effect on the 10-min laptop PVT than We are grateful to the staff and research assistants of the Sleep and Performance Research Center at Washington State University Spokane for their help conon the 5-min PDA PVT. Even so, both versions of the PVT cap- ducting the study, and to Hans Van Dongen for guidance on research design tured the essence of the RT distribution changes that and execution and statistical analysis. We would like to thank Dr. Matthew characterize the effects of sleep deprivation on psychomotor Layton, the physician of record for the study. This study was supported by the Boeing Company. vigilance performance.5,9,19 Authors and affiliation: Kimberly A. Honn, Ph.D., Samantha M. Riedy, B.S., and Devon A. Grant, M.S., Human Sleep and Cognition Laboratory, Sleep and Performance Research Center, Washington State University, Spokane, WA.
REFERENCES
Fig. 4. Means and standard errors for KSS sleepiness scores reported hourly across 38 h of TSD in the laboratory. The KSS scale ranges from 1 (“extremely alert”) to 9 (“extremely sleepy – fighting sleep”). Plotted values control for order of administration of the two PVT versions.
1. Åkerstedt T, Anund A, Axelsson J, Kecklund G. Subjective sleepiness is a sensitive indicator of insufficient sleep and impaired waking function. J Sleep Res. 2014; 23(3):242–54. 2. Åkerstedt T, Gillberg M. Subjective and objective sleepiness in the active individual. Int J Neurosci. 1990; 52(1-2):29–37. 3. Balkin TJ, Bliese PD, Belenky G, Sing H, Thorne DR, et al. Comparative utility of instruments for monitoring sleepiness-related performance decrements in the operational environment. J Sleep Res. 2004; 13(3):219–227. 4. Basner M, Dinges DF. Maximizing sensitivity of the psychomotor vigilance test (PVT) to sleep loss. Sleep. 2011; 34(5):581–591.
AEROSPACE MEDICINE AND HUMAN PERFORMANCE Vol. 86, No. 5
May 2015
433
PORTABLE PVT VALIDATION—Honn et al.
5. Basner M, Mollicone D, Dinges DF. Validity and sensitivity of a brief 19. Ratcliff R, Van Dongen HPA. Diffusion model for one-choice reactionPsychomotor Vigilance Test (PVT-B) to total and partial sleep deprivatime tasks and the cognitive effects of sleep deprivation. Proc Natl Acad tion. Acta Astronaut. 2011; 69(11-12):949–959. Sci U S A. 2011; 108(27):11285–11290. 6. Basner M, Rubinstein J. Fitness for duty: a 3-minute version of the 20. Roach GD, Dawson D, Lamond N. Can a shorter psychomotor vigilance Psychomotor Vigilance Test predicts fatigue related declines in luggage task be used as a reasonable substitute for the ten-minute psychomotor screening performance. J Occup Environ Med. 2011; 53(10):1146–1154. vigilance task? Chronobiol Int. 2006; 23(6):1379–1387. 7. Chee MWL, Chuah LYM, Venkatraman V, Chan WY, Philip P, Dinges DF. 21. Selya AS, Rose JS, Dierker LC, Hedeker D, Mermelstein RJ. A practical Functional imaging of working memory following normal sleep and after guide to calculating Cohen’s f2, a measure of local effect size, from PROC 24 and 35 h of sleep deprivation: correlations of fronto-parietal activation MIXED. Front Psychol. 2012; 3:111. with performance. Neuroimage. 2006; 31(1):419–428. 22. Thorne DR, Johnson DE, Redmond DP, Sing HC, Belenky G, Shapiro 8. Cohen JE. Statistical power analysis for the behavioral sciences, 2nd rev. JM. The Walter Reed palm-held psychomotor vigilance test. Behav Res ed. Hillsdale (NJ): Lawrence Erlbaum Associates; 1988:410–414. Methods. 2005; 37(1):111–118. 9. Dinges DF, Kribbs NB. Performing while sleepy: effects of experimentally23. Tucker AM, Basner RC, Stern Y, Rakitin BC. The variable responseinduced sleepiness. In: Monk TH, editor. Sleep, sleepiness, and perforstimulus interval effect and sleep deprivation: an unexplored aspect of mance. New York: Wiley; 1991:97–128. psychomotor vigilance task performance. Sleep. 2009; 32(10):1393–1395. 10. Dinges DF, Powell JW. Microcomputer analyses of performance on a 24. Van Dongen HP, Belenky G, Krueger JM. A local, bottom-up perspective portable, simple visual RT task during sustained operations. Behav Res on sleep deprivation and neurobehavioral performance. Curr Top Med Methods Instrum Comput. 1985; 17(6):652–655. Chem. 2011; 11(19):2414–2422. 11. Doran SM, Van Dongen HPA, Dinges DF. Sustained attention performance 25. Van Dongen HPA, Belenky G, Krueger JM. Investigating the temporal during sleep deprivation: evidence of state instability. Arch Ital Biol. 2001; dynamics and underlying mechanisms of cognitive fatigue. In: Ackerman 139(3):253–267. PL, editor. Cognitive fatigue: multidisciplinary perspectives on current 12. Dorrian J, Rogers NL, Dinges DF. Psychomotor vigilance performance: research and future applications. Washington (DC): American Psychoneurocognitive assay sensitive to sleep loss. In: Kushida CA, editor. Sleep logical Association; 2011:127–147. deprivation: clinical issues, pharmacology, and sleep loss effects. New 26. Van Dongen HPA, Dinges DF. Sleep, circadian rhythms, and psychomotor Delivered by Ingenta to: Purduevigilance University Libraries York: Marcel Dekker; 2005:39–70. . Clin Sports Med. 2005; 24(2):237–249. IP:GD, 79.110.17.193 On:onWed,27.03Van Aug 2016 02:31:15 13. Ferguson SA, Paech GM, Dorrian J, Roach Jay SM. Performance Dongen HPA, Maislin G, Dinges DF. Dealing with inter-individual Copyright: Aerospace Medical Association a simple response time task: is sleep or work more important for miners? differences in the temporal dynamics of fatigue and performance: imAppl Ergon. 2011; 42(2):210–213. portance and techniques. Aviat Space Environ Med. 2004; 75(3, Suppl.): 14. Kribbs NB, Dinges DF. Vigilance decrement and sleepiness. In: Harsh A147–A154. JR, Ogilvie RD, editors. Sleep onset mechanisms. Washington (DC): 28. Van Dongen HPA, Maislin G, Mullington JM, Dinges DF. The cumulative American Psychological Association; 1994:113–125. cost of additional wakefulness: dose-response effects on neurobehavioral 15. Krueger C, Tian L. A comparison of the general linear mixed model and functions and sleep physiology from chronic sleep restriction and total repeated measures ANOVA using a dataset with multiple missing data sleep deprivation. Sleep. 2003; 26(2):117–126. points. Biol Res Nurs. 2004; 6(2):151–157. 29. Van Dongen HPA, Mollicone DJ. Field study on the efficacy of the new 16. Lim J, Dinges DF. A meta-analysis of the impact of short-term sleep restart provision for hours of service. Washington (DC): Federal Motor deprivation on cognitive variables. Psychol Bull. 2010; 136(3):375–389. Carrier Safety Administration; 2013. Report No: RRR-13-058. 17. Lim J, Dinges DF. Sleep deprivation and vigilant attention. Ann N Y Acad 30. Wesensten NJ, Belenky G, Thorne DR, Kautz MA, Balkin TJ. Modafinil Sci. 2008; 1129:305–322. vs. caffeine: effects on fatigue during sleep deprivation. Aviat Space 18. Petrilli RM, Roach GD, Dawson D, Lamond N. The sleep, subjective Environ Med. 2004; 75(6):520–525. fatigue, and sustained attention of commercial airline pilots during an 31. Winer BJ. Statistical principles in experimental design, 2nd ed. New York: international pattern. Chronobiol Int. 2006; 23(6):1357–1362. McGraw Hill; 1971:378–384.
434
AEROSPACE MEDICINE AND HUMAN PERFORMANCE Vol. 86, No. 5
May 2015
