Edited by: Nadia Bianchi-Berthouze, University College London, UK
Reviewed by: Gualtiero Volpe, Università degli Studi di Genova, Italy; Rubén San Segundo Hernández, Universidad Politécnica de Madrid, Spain; Sandy J. J. Gould, University College London, UK
*Correspondence: Harry J. Witchel
This article was submitted to Human-Media Interaction, a section of the journal Frontiers in Psychology
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
Instrumental movements are fundamental to the process of the task at hand; for a person interacting with a computer, they include using the mouse (or any controller such as a keyboard), postural actions required to use the controller (e.g., leaning forward), and head and eye movements that are used for targeting gaze. Non-instrumental movements are not task-required, e.g., fidgeting, scratching, stretching, and emotional expressions; although not task-required, they are often unwittingly task-induced via cognitive states (Ekman and Friesen,
The rationales for linking assessments of nonverbal behavior (such as task-induced non-instrumental movements and gestures) to cognitive states are two-fold: (1) there is a long-standing scientific literature on nonverbal behavior and its meaning (Bull,
Recognition in this way will be important in responsive learning systems such as automated tutors (D'Mello et al.,
Postural movements, in particular, including movements of the head (D'Mello et al.,
Some folk psychology theories (Pease and Pease,
Our team recently demonstrated that Non-Instrumental Movement Inhibition (NIMI) could manifest as a marker for cognitive engagement in seated computer users (Witchel et al.,
In the psychology literature, many gestures and non-instrumental movements during dyadic communication have been associated with increasing engagement; for example, a sitting speaker is over 10 times more likely to draw their legs backward when speaking on an interesting topic than when bored (Bull,
There is tacit disagreement in HCI over the interpretation of increased levels of movement. Bianchi-Berthouze and colleagues, testing standing game players, showed that task-related movement results in greater subjective engagement (Bianchi-Berthouze et al.,
For example, Kapoor et al. (
This claim relies on being able to distinguish instrumental from non-instrumental movement; for example, one can minimize instrumental movements using a handheld trackball (which only requires finger activity, and does not interfere with free arm or shoulder movement) to eliminate instrumental movements except those associated with gaze and with the fingers. However, continuous stimuli used to compare engagement to disengagement do not tend to control for interaction rate or instrumental movement in their analyses of movement, and thus previous studies have not differentiated instrumental movements from non-instrumental movements. Thus, in video games or intelligent tutor examples (Mota and Picard,
Engagement and attention are related but not equivalent. Attention can mediate instantaneously between many competing stimuli, while engagement lasts longer, is not as completely exclusive, and implies at least a partial commitment to action or volition (Henrie et al.,
Because engagement is longer in duration and not completely exclusive, it can be defined as a family of related cognitive states geared toward extended interaction and/or a purposeful outcome, operationalized by a collection of behaviors, none of which are absolutely necessary at a given point in time, including: attendance, attention, memory, caring, emotion, taking action, making an effort, and (like the exclusion in attention) inhibition of irrelevant activities (Witchel,
Most commentators on engagement subdivide its manifestations into categories such as cognitive, emotional/affective, and physical/behavioral/motor engagement (Bloom,
The total movements of any body part is a mixture of instrumental and non-instrumental movements, and the non-instrumental movements of the head will be limited by the instrumental needs of where a person needs to look. For example, a person watching a television has to face the screen in order to fully engage with the visual content, while a person listening to the radio can face in any direction and still listen to (and fully engage with) the radio content. Hypothetically, targeting the gaze to watch something on a fixed-position screen should concurrently suppress some non-instrumental head positions and movements (i.e., it elicits NIMI) compared to behaviors elicited by comparable audio-only content.
In addition to position, many body parts' velocity and speed will be affected by engagement; for example, reading text is quite difficult when nodding one's head. By contrast, thigh movements are not intimately connected to gaze. One can easily read a book while seated even when the legs are in constant motion (e.g., jigging the leg). Thus, thigh movement is not of necessity instrumentally restricted by the process of reading or screen watching.
Testing for the relationship between stimulus-elicited emotions and their manifestations is more relevant for applications when using continuous stimuli, but it should be more experimentally tractable when using short, discrete stimuli because the elicited emotions will be more predictably homogeneous. In continuous stimuli, such as long video game playing sessions, it is assumed that the end-user's emotional state varies despite the stimulus being relatively uniform (i.e., due to boredom or fatigue). In experiments using continuous stimuli, ground truth for emotions is provided either by (a) interrupting play to sample emotions, (b) asking the end-user to review films of himself/herself, or (c) asking experts to interpret films of the end-user's body activity. These techniques either assume that the player has clear insight into their own emotions, or that the nonverbal behavior interpretations of experts are true. Since part of the goal of this study was to test the current assumptions of nonverbal behavior experts, we settled on using discrete stimuli.
For brief stimuli (e.g., using the International Affective Photographic System; Lang and Bradley,
In study 1, we sought conclusive data to support the hypothesis that adding visual content to an audio-only stimulus will decrease head movements (but not necessarily thigh movements), irrespective of whether the subjective engagement in the multimodal stimulus is modestly higher or lower than during the audio stimulus. That is, so long as the viewer makes an effort to attend to the screen (whether or not they feel the content is engaging), they will suppress large non-instrumental movements that might cause their head to face away from the screen; by contrast, audio-only stimuli (even when engaging) allow for large non-instrumental head movements. This experiment will support the idea that attention and targeting gaze has an inhibitory effect on non-instrumental head movements.
In study 2, we designed stimuli to test directly whether cognitive (rather than physical) engagement itself is responsible for lowering non-instrumental movement in the body generally, independent of interaction rate. We created special reading comprehension test stimuli with high interaction rates (27 clicks per min), while instrumental movements were minimized by mediating all interactions via a handheld trackball. Interest should be sufficient for reducing head and postural movements, which we term NIMI. Also, in both studies we tested the hypothesis that engagement is associated with the seated computer user approaching the screen, with measurements of mean head distance from the screen.
Both studies were organized using a single independent variable with three levels; the independent variable being tested was the stimulus. The rationale for using three levels to test one hypothesis was to support the hypothesis with two comparable stimuli, but also for the third stimulus to provide an exception to the oversimplified hypothesis that cognitive engagement always diminishes total movement. In both studies the third stimulus was highly engaging while varying the visual demands upon the viewer, in order to demonstrate the dangers of conflating non-instrumental movement with total movement. Thus, the third stimulus in each study demonstrates the need for the NIMI concept to avoid this conflation, by exemplifying the apparent exceptions to the oversimplified hypothesis.
There are several theoretical causes for a seated person interacting with a computer to move less: targeting gaze and attention, rapt engagement, increased mouse/keyboard interactions (or other instrumental actions that lock the shoulder in place), and lethargic boredom. As a precursor to testing the effect of cognitive engagement on movement in study 2, we verified in study 1, that attractive and relevant on-screen visual stimuli can lead to diminished movement. To test this hypothesis, we used two passive (i.e., non-interactive) stimuli that were similar in terms of audio. It may seem self-evident that having something to look at will cause the head to move less, but this implies that being engaging
Twenty-seven healthy participants (15 women, 12 men; mean age 21.00,
The single independent variable was stimulus, of which there were three: FAV (having no visual component), OK Go video (multimodal), and OK Go audio-only. The purposeful planned comparison (video vs. audio-only for the OK Go songs) was meant to be accompanied by the important exception: that highly engaging audio-only FAV was expected to elicit more head movement than less engaging video stimuli. The primary dependent variables were subjective response (Visual Analog Scale) for “I felt totally engaged,” and the speed of movement of the reflective motion capture markers on the body.
We have designed these experiments to be repeated measures comparisons, despite the fact that two different songs by OK Go were used in each of the conditions (video vs. audio-only). There is a precedent for grouping different pieces of music as a single type of stimulus; when scientists need to elicit strong responses to music, they ask for user-selected favorite music and analyze it as a group (Blood and Zatorre,
When considering what drives the responses to different songs, there are four relevant factors: the song itself, the medium (video vs. audio-only), consistent issues relating to the individual participant, and an error factor. The design of a repeated measures analysis relates to the planned consistencies in participant factors, which are quite strong in our experiments. For example, in subjective measures there are consistencies within individual participant measurements, such as how interested they are in taking part in a psychological experiment (which would increase ratings of engagement for most stimuli). Likewise, how fidgety a participant is can affect all their head movements.
After each 3-min stimulus, participants completed both the Self-Assessment Manikin (SAM) and a collection of Visual Analog Scales (VAS); these are 10 cm rating scales with anchors at 0 (“Not At All”) and at 100 (“Extremely”). The scales were “I wanted to see/hear more,” “I felt totally engaged,” “I felt interested.” “I wanted it to end earlier,” “I felt bored,” “I felt frustrated.” In this study, we have operationalized engagement by asking the participants to subjectively assess their own engagement.
The SAM (Bradley and Lang,
The methodology of this kind of study has been described previously (Witchel et al.,
The experimental set up is shown in Figure
Digital films of body movement were captured as 3 min fragments for each stimulus using Movie Maker on computers running Windows. The motions of all reflective markers in two dimensions were tracked using Kinovea 1.0, and two-dimensional known length standards (10 cm QR codes that can be recognized by computer) were used to calibrate the Kinovea measurements. These two-dimensional measurements of movements were previously shown to be highly correlated with three-dimensional movements captured with the gold standard (Vicon opto-electronic motion capture system; Witchel et al.,
The parsing of each time course into an 82-s segment is described in full in (Witchel et al.,
Statistical analyses of subjective and movement data were performed in Matlab.
Passive stimuli were audiovisual stimuli without the requirement for interaction (e.g., mouse activity). Each stimulus lasted 175 s, with source videos being cut-short to fit our format. Two similarly engaging music videos by the band OK Go (Supplementary Figure
One experimental design issue is that for a comparison of a music video with and without the video in a paired design, the same song could not be experienced twice because the second music video stimulus would be subject to habituation and boredom. To avoid this we used two popular videos made by the same band (OK Go) that elicited very similar levels of subjective engagement (see Tables
Song 1: Here It Goes Again (HIGA) | Multimodal | Moderately engaging |
Song 1: Here It Goes Again (HIGA) | Audio only | Partially engaging |
Song 2: Do What You Want (DWYW) | Multimodal | Moderately engaging |
Song 2: Do What You Want (DWYW) | Audio only | Partially engaging |
Music: FAV | Audio only | Very engaging |
Audio only: Pooled OK Go | 32.78 (21.05) |
Audio only: Here It Goes Again | 34.64 (15.99)† |
Audio only: Do What You Want | 30.77 (25.97)* |
Multimodal: Pooled OK Go | 61.11 (17.83) |
Multimodal: Here It Goes Again | 60.38 (19.41)* |
Multimodal: Do What You Want | 61.79 (16.94)† |
Audio Only: FAV (a favorite piece) | 71.85 (20.62) |
As expected, there were no significant differences in engagement between the two multimodal stimuli (i.e., as music videos) or the two audio-only stimuli (i.e., as songs, with the computer screen being completely black); that is, the songs Here It Goes Again (HIGA) and Do What You Want (DWYW) did not differ in engagement (see Table
Multimodal | HIGA | Multimodal | DWYW | –20.94 | –1.41 | 18.38 | 0.9981 |
Audio only | HIGA | Audio only | DWYW | –16.76 | 3.88 | 22.56 | 0.9795 |
Multimodal | HIGA | Audio only | HIGA | 7.06 | 25.73 | 46.38 | 0.0038 |
Multimodal | DWYW | Audio only | DWYW | 11.24 | 31.02 | 50.56 | <0.001 |
Subjectively, the outright loss of visual content in the matched music videos made the stimulus less engaging; the subjective responses for VAS engaged were significantly different between multimodal vs. audio-only stimuli [see Table
Audio only | Multimodal | –39.72 | –28.33 | –16.95 | <0.001 |
Music: FAV | Audio only | 27.69 | 39.07 | 50.46 | <0.001 |
Music: FAV | Multimodal | –0.64 | 10.74 | 22.12 | 0.0682 |
Audio only | Multimodal | –3.81 | 6.19 | 16.18 | 0.3023 |
Music: FAV | Audio only | –27.47 | –17.48 | –7.49 | <0.001 |
Music: FAV | Multimodal | –21.29 | –11.30 | –1.31 | 0.0232 |
Listening to preferred music, can elicit a range of non-instrumental movements in the listener, often at a subconscious level (Witchel,
The three musical stimuli (FAV, audio-only, multimodal) differed highly significantly in terms of elicited head movement [see Table
Multimodal | 0.29 (0.31) | 0.06 (0.06) |
Audio only | 0.88 (0.88) | 0.14 (0.12) |
Music: FAV | 1.09 (0.92) | 0.23 (0.35) |
Audio only | Multimodal | 0.26 | 0.60 | 0.94 | <0.001 |
Music: FAV | Multimodal | 0.46 | 0.80 | 1.14 | <0.001 |
Music: FAV | Audio only | –0.13 | 0.20 | 0.54 | 0.3199 |
Audio only | Multimodal | –0.06 | 0.08 | 0.22 | 0.3722 |
Music: FAV | Multimodal | 0.03 | 0.17 | 0.31 | 0.0139 |
Music: FAV | Audio only | –0.05 | 0.09 | 0.23 | 0.2662 |
Thigh movements differed significantly between FAV and the multimodal stimulus, but not between the two OK Go stimuli [audio-only and multimodal, see Table
In terms of mean head distance from the screen (i.e., position, rather than movement), there was a trend (paired
This study supports the hypothesis that visual stimuli reduce head movement, and it provides an exception to the hypothesis that engagement reduces total movement. As expected, when adding appropriate video content to the OK Go songs, the resulting stimuli were more engaging while reducing head movement. This reduction in movement could be due to either needing to gaze at the monitor or to increased engagement. FAV (another audio-only stimulus) also elicited much more head movement than the multimodal stimulus. This result for FAV plainly violates the heuristic that, when seated, increased total movement implies lower engagement.
The two audio-only stimuli (FAV and audio-only OK Go) elicit much more movement than the multimodal OK Go; of the two audio-only stimuli, FAV is more subjectively engaging than the multimodal OK Go video, while the audio-only OK Go song is less engaging. This implies that, for these examples, engagement is less important in determining the amount of elicited movement than whether there is visual accompaniment, and potentially how persistently, the viewer needs to watch this, depending on whether the visual content is challenging, demanding or time-sensitive.
The difficulty of relating subjective engagement ratings to movement during non-visual musical stimuli is highlighted by our previous data showing that highly disengaging music elicits even more non-instrumental movement than favorite music in healthy male volunteers (Witchel et al.,
The results of previous experiments that demonstrated an association between engagement and reduced movement often concurrently showed lower levels of engagement in association with lower levels of user interaction (van den Hoogen et al.,
Study 2 was designed as a single independent variable with three levels, each being a different stimulus: two specially constructed, interactive reading stimuli, and a game (see Stimuli Section). This study was conducted with the same participants (and in the same hour-long session) as study 1, using identical instruments and scales as in study 1, as well as identical data analysis and movement measurements, and an identical procedure. Because study 2 included a commercial game (Zuma, see Stimuli Section), participants who had never played Zuma before were instructed in how to play, and allowed to play for 3 min before any measurements were made, in order to prepare them for the experimental playing session later, and to familiarize them with the use of the handheld trackball.
In this study, three time-sensitive, interactive stimuli were used: a commercial video game called Zuma (stimulus abbreviation ZU, in which the player has to shoot colored balls at other rolling balls that match its color before the rolling balls reach the finish line), and two reading comprehension tests made in Macromedia Flash Professional 8. These interactive stimuli are summarized in Table
EU regulations: EUB | 27 | Boring |
Best seller: CIDN | 27 | Engaging |
Game: ZUMA | 30–60 | Highly engaging |
To vouchsafe that the reading tasks had a constant amount of interaction, approximately every 2 s (at inconsistent intervals) the reading was replaced with a gray screen, which remained in place until the user clicked anywhere on the screen with the handheld trackball, after which the reading returned. Volunteers were instructed to click as quickly as possible when they saw the gray screen, as otherwise, they might miss some of the text. This appearing and disappearing feature was described by many participants as slightly irritating, as it kept them on edge during the reading task. This feature meant that the interaction rate for the reading tasks was comparable to the interaction rate of Zuma.
Subjective ratings for the reading comprehension tests (Descriptive statistics see Table
EU regulations: EUB | 23.93 (19.38) | 69.44 (30.73) |
Best seller: CIDN | 61.67 (23.41) | 46.30 (34.52) |
Game: ZUMA | 77.41 (17.23) | 22.78 (23.30) |
EU Regs: EUB | Best Seller: CIDN | –48.47 | –37.74 | –27.01 | <0.001 |
Game: ZUMA | EU Regs: EUB | 42.75 | 53.48 | 64.21 | <0.001 |
Game: ZUMA | Best Seller: CIDN | 5.01 | 15.74 | 26.47 | 0.0024 |
EU Regs: EUB | Best Seller: CIDN | 5.80 | 23.15 | 40.50 | 0.0062 |
Game: ZUMA | EU Regs: EUB | –64.02 | –46.67 | –29.32 | <0.001 |
Game: ZUMA | Best Seller: CIDN | –40.87 | –23.52 | –6.17 | 0.0053 |
In terms of movement elicited, the three interactive stimuli (best-seller CIDN, EU regulations EUB and game ZU) differed highly significantly in terms of elicited head movement [see Table
EU regulations: EUB | 0.41 (0.41) | 0.12 (0.11) |
Best seller: CIDN | 0.24 (0.25) | 0.07 (0.07) |
Game: ZUMA | 0.30 (0.19) | 0.07 (0.05) |
EU regs: EUB | Best seller: CIDN | 0.03 | 0.17 | 0.32 | 0.0140 |
Game: ZUMA | EU regs: EUB | –0.26 | –0.11 | 0.03 | 0.1389 |
Game: ZUMA | Best seller: CIDN | –0.08 | 0.06 | 0.20 | 0.5889 |
EU regs: EUB | Best seller: CIDN | 0.01 | 0.05 | 0.09 | 0.0148 |
Game: ZUMA | EU regs: EUB | –0.10 | –0.05 | –0.01 | 0.0128 |
Game: ZUMA | Best seller: CIDN | –0.04 | 0.00 | 0.04 | 0.9983 |
The interactive reading comprehension quizzes elicited significantly different head movement speeds from each other; the mean forehead speed for the boring EU reading test (EUB) was 72% faster than the engaging best-seller (CIDN) reading test (see Table
However, increased engagement (or decreased frustration) does not necessarily lead to significant NIMI (i.e., lower head movement speeds); the most engaging interactive stimulus (the commercial game ZU) did not elicit significantly different head movement than either of the reading tests (see Tables
The elicited thigh movements differed significantly between the boring stimulus (EUB) and the interesting ones (ZU and CIDN), but not between the two interesting stimuli [see Table
In terms of mean head distance from the screen (i.e., position, rather than movement), there was a significant difference between the engaging game ZU and the interesting reading test (CIDN), but not between any other pairs of interactive stimuli [Table
EU regs: EUB | Best seller: CIDN | –0.51 | 1.00 | 2.52 | 0.2555 |
Game: ZUMA | EU regs: EUB | –0.80 | 0.71 | 2.23 | 0.4970 |
Game: ZUMA | Best seller: CIDN | 0.20 | 1.72 | 3.23 | 0.0229 |
The hypothesis tested in this study was that seated participants decrease their movements in response to more engaging interactive video experiences. EUB and CIDN have precisely identical interaction rates, while ZU has a comparable interaction rate. ZU was included to demonstrate potential exceptions to the rule due to the differences between total movements vs. non-instrumental movements.
The tested hypothesis was strongly supported by the matched reading comprehension quizzes (nearly a two-fold difference in head and thigh movement,
In our speed measurements of total head movement, engaging ZU's elicited instrumental head movement speed approaches boring EUB's non-instrumental head movement speed. By contrast, the mean thigh movement levels of ZU are nearly the same as the engaging CIDN. During seated HCI there is rarely an instrumental reason for the participant to move the thigh. This is why measurements of total thigh movements may reveal a difference between engaging and boring stimuli. It should be noted that the thigh makes much less movement than the head, and often it does not move at all during the course of 82 s.
In this study we chose not to include our shoulder measurements because the head and shoulders reflect similar (but not identical) movement—in particular, whenever the shoulders make movements forward or backward, the head usually moves the same way because the neck's base is connected to the shoulders. We did not measure foot (or hand) movement for several technical reasons. The fact, that the arm and leg can rotate (e.g., supination and pronation) means that for our camera-based set up there will be problems with occlusion of the markers, which would create discontinuous data. This occlusion problem is worsened by furniture. The potential solution is to use wearable inertial sensors, which will provide clear indicators of movement, and under good conditions relatively precise readings of position.
Speed (rate of change) is a metric that has emerged in our lab and others (D'Mello et al.,
There are many possible features, including acceleration. jerk, standard deviation, kurtosis, skew, entropy, and spectral features (comparing different ranges outputted from Fourier transforms) including spectral energy and the amount of white vs. pink noise (D'Mello et al.,
In this paper two studies both involved the use of three stimuli to investigate the claimed hypothesis that video engagement can be recognized by diminished postural movement and that boredom and frustration are associated with more movement. There is currently no way to distinguish instrumental from non-instrumental movements based on the movement records alone. Our approach was, therefore, to design stimuli and interactions that minimized instrumental movements (i.e., by using a handheld trackball instead of a mouse), so that the only instrumental movements were head movements related to the targeting of gaze (and very small finger movements associated with the trackball). We found evidence to support three major conclusions:
Conclusion (1) The primary hypothesis is supported. To the best of our knowledge, study 2 is the clearest example showing that when stimuli are directly comparable (e.g., matched interactivity rates), cognitive engagement is associated with an inhibition of non-instrumental movements. Conclusion (2) Head movements associated with the targeting of gaze can make a profound difference to the movement results that one detects, and that apparent exceptions to the primary hypothesis can be found if one does not consider instrumental head movements associated with the targeting of gaze. That is, inhibition of head movement is more strongly driven by the need to watch the screen than by cognitive engagement. Conclusion (3) As a corollary to the above findings, we presented evidence that when people are seated, thigh movement seems to be inhibited during engagement. Thus, NIMI can affect parts of the body that are not necessarily instrumental in gaze targeting. Therefore, NIMI is not just an epiphenomenon of visual attention—it relates to cognitive engagement
The novel additions of this study to the literature are: (A) the two reading comprehension stimuli in study 2 (EUB and CIDN) are absolutely comparable; they are the same stimulus except that the words are different, and this difference in words is enough to change both how interesting the visual stimulus is, and how much movement it elicits. (B) The two reading comprehension stimuli in study 2 are highly interactive (27 mouse clicks per min); this means that during the boring stimulus (EUB) the participants were looking at the stimulus, countering the trivial explanation that they were looking around the room rather than looking at the screen. (C) The trivial explanation (if you do not have to look at the screen, you can move your head more) is clearly demonstrated in study 1, and we show that looking around the room elicits much more head movement (mean speed > 0.88 mm/s) than even the most boring stimulus that requires consistent visual attention (EUB mean speed = 0.41 mm/s); the interesting visual stimuli elicited even lower head speeds.
Thus, study 1 shows that increased cognitive engagement is neither necessary nor sufficient to diminish total movement. There are other factors that diminish movement including targeting gaze and attention, increased mouse/keyboard interactions (or other instrumental actions that lock the shoulder in place), and lethargic boredom. The likely factors that increase movement will be high arousal, break-taking, frustration and suppressed escape, emotional expression, and instrumental activity. Furthermore, the development of the universally boring reading comprehension test (EUB) demonstrates that it is possible to have a high interaction rate while being subjectively boring and not engaging; thus, interactivity is not synonymous with cognitive engagement. We theoretically synthesize the conflicting observations from the two literatures mentioned in the introduction (i.e., in museums and standing game-play more movement implies engagement while in HCI more movement implies boredom or frustration) as follows: (1) physical engagement alters instrumental movement, (2) physical engagement tends to cause cognitive engagement, and (3) purely cognitive engagement with fixed screens tends to cause NIMI. Key to progress in this field will be the ability to computationally distinguish instrumental from non-instrumental movements; this may occur with careful analysis of the structure of movement (D'Mello et al.,
Finally, the link between sitting forward and cognitive engagement continues to defy explanation. This study provides two more examples failing to confirm that engagement leads to a forward head position
HW oversaw the experiments, designed the experiments, and drafted the first manuscript. CS performed most of the experiments, and contributed to the manuscript's completion. JA conceived of the original idea for study 1, and contributed to the manuscript. CW conceived of the original idea for study 2 and its stimuli. She made major corrections to the manuscript, and made several figures plus the video. NC oversaw the technical side of the measurements, including the original work with video tracking.
This research was partially funded by a Wellcome Trust Biomedical Vacation Scholarship to CS (105298/Z/14/Z), and in part by the Independent Research Project program run by BSMS to JA and HW.
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. The reviewer, SG, and handling Editor declared their shared affiliation, and the handling Editor states that the process nevertheless met the standards of a fair and objective review.
We gratefully acknowledge the contributions of Jacob Greaves for careful and extensive data analysis, Chätrin Tolga for volunteer coordination and digital capturing of the films, and Pete Docter for the original ideas to capture emotions with movement.
The Supplementary Material for this article can be found online at: