The Influence of Target Layout and Clicking Method on Picking Time and Dragging Performance Based on Eye-Control Technique

Eye-tracking has been a hot topic in human–computer interaction (HCI). Nevertheless, previous studies usually adopted eye-tracking as information output rather than input. The eye-control technique can achieve convenient and rapid real-time operation through the movement of the eyes and reduce unnecessary manual operations. Because the layout determines the location orientation, organizational complexity, cognitive consistency, and predictive ability of the information display, the interface layout design affects the user’s perception of information intensity, complexity, and logic. Moreover, the method of target clicking by eye-control techniques, which include blink and dwell, also depends on the application and user’s ability. The purpose of this study is to investigate the influence of target layout and target picking method on picking time and dragging performance based on eye-control technique. The results indicate that the target picking method, i.e., blink or dwell, had significant effects on the dragging time and dragging numbers. However, there was no significant effect of target layout on picking time and dragging performance (dragging time and numbers), which may be related to the setting of the experimental conditions (e.g., lighting level and screen resolution). Moreover, the target picking method and the target layout had no significant interaction effect on picking time and dragging performance. The findings are anticipated to provide helpful implications for future eye control technique design.

Introduction

Eye-tracking technology has spread widely in the last decade, but it is seldom reported that eye-control techniques are used in either academia or industrial applications. Eye control is an advanced technique in human–computer interaction (HCI) research that can achieve convenient and rapid real-time operation through the movement of the eyes and reduce unnecessary manual operations. With the deepening of eye movement sensing and pattern recognition, this technology is most widely used in education- (Wu, 2012), medicine- (Harezlak and Kasprowski, 2018), military- (Muehlethaler and Knecht, 2016), entertainment- (Lin et al., 2004), and psychology-related fields (Renshaw et al., 2003).

There are some different types of eye movement that were identified by previous researchers, some of which keep the fovea on a visual target in the environment (e.g., saccades and smooth pursuits), while others stabilize the eye during head movement (e.g., fixations) (Underwood and Radach, 1998; Duchowski, 2007; Liversedge et al., 2011). Through these different types of eye movements, the eye can complete the aiming and continuous dynamic observation of the object, thus ensuring clear visual input. The eye tracker is a complex and precise psychological instrument. It can measure individual eye movement characteristics and evaluate the validity of effective interfaces by visual cognitive physiological evaluation (Zhao et al., 2018). Eye tracking techniques have been applied to investigate HCIs (Jacob and Karn, 2003; Goldberg and Helfman, 2011).

In graphical user interaction, spatial attributes (such as topology, geometry, spatial relationships, etc.) are mapped to functional attributes (such as causality, hierarchical relationships, associations, etc.). For example, Liu et al. (2016) pointed out that stable covariant structural information can reduce the complexity of the scene while increasing its predictability. Because the layout determines the location orientation, organizational complexity, cognitive consistency, and predictive ability of the information displayed, the interface layout design affects the user’s perception of information intensity, complexity, and logic. In the 1950s, Fitts et al. (1949, 1950) studied the series of eye movements in the pilot landing process to determine an effective method of assessing the importance of the instrument, the difficulty of instrument reading, and the instrument layout design by eye-moving techniques. Eye movements are thought to provide an indication of the amount of cognitive processing display requirements (Raynor and Pollatsek, 1994), and eye tracking can be a tool for the assessment of usability (Renshaw et al., 2003; Rayner, 2009). Pušnik et al. (2016) studied how layout, typeface use, position of titles and/or text, color combination draw attention and affect the recall of presented content by using eye-tracking technology. Nevertheless, previous studies usually adopted eye-tracking as information input rather than eye control as information output. How users interact with the graphic user interface (GUI) and what kinds of GUI parameters influence the interaction performance deserve deep investigation.

Because the layout determines the location orientation, organizational complexity, cognitive consistency, and predictive ability of the information display, the interface layout design affects the user’s perception of information intensity, complexity, and logic. Moreover, the method of target clicking by eye-control techniques, which include blink and dwell, also depends on the application and user’s ability. Thus, we proposed the hypotheses as follows:

H1: The increase of operation repetitions had significant effects on the performance.

H2: The target layout had significant effects on the dragging time and dragging numbers.

H3: The target picking method (blink or dwell) had significant effects on dragging performance.

H4: The target picking method and the target layout had no significant interaction effect on picking time and dragging performance.

This paper aimed to investigate the influence of target layout and target clicking method on picking time and dragging performance based on eye-control technique. Through the study of visual perception and information processing mode, we explored whether the position on the screen and the target clicking method would affect the operation process. The results could provide some suggestions for the design of human-machine interfaces and clicking methods that are more suitable for eye-control systems.

Materials and Methods

Experiment Design

We performed a repeated measures experiment to investigate the influence of target layout and target clicking method on picking time and dragging performance based on eye-control technique. The independent variables were the initial position of the target, which had five values (screen center, top left, bottom left, top right, and bottom right), and the method of target picking, which had two values: blink (blinking twice as a click) and dwell (focusing the eyes on the target for 1 s as a click). The dependent variables were the picking time (the time of picking the target successfully), the dragging time (the time of dragging the target into the specified range and dropping successfully), and the dragging numbers (the number of targets successfully dragged the target into the specified range). The last two variables were considered variable indicators of dragging performance.

Task Design

The target square has five positions: screen center, top left, bottom left, top right, and bottom right. A set of a target square (one of the five positions) and a circular target (around the target square) is randomly displayed on the screen. The interactive interface is shown in Figure 1A. The participants were asked to use eyeball movement to drag the target into the specified circular range and drop the target. The operation of successfully putting the target square into the circular range is shown in Figure 1B. The participants completed the target picking and dropping by two different methods, blink (blinking twice as a click) and dwell (focus the eyes on the target for 1 s as a click). The interactive interface of blink and dwell is shown in Figures 1C,D.

FIGURE 1

Figure 1. The experimental user interfaces. (A) Initialization interface. (B) Putting the target square into the circular range. (C) Interactive interface of blink. (D) Interactive interface of dwell. Reproduced with permission from Springer Nature.

Participants

In the experiment, we recruited 16 participants between 20 and 27 years old, with an average age of 22 years. The participants were university students and working engineers from the Institute of Marine Technology and Economy, China. All participants had not been familiar with the experiment in advance, and all participants were required to maintain normal eyesight without glasses. We randomly select the participant to complete the experiment independently.

Experimental Facility

We chose the Tobii Eye Tracker 4C for eye movement control which can track the eye and head at the same time. The parameters of the eye tracker are as follows: operating distance: 20–37”/50–95 cm, and image sampling rate: 90 Hz. The working principle of the eye tracker is: the projector projects the pattern into the eye through near infrared rays, the cameras take high-resolution images of the user’s eyes and pattern, the image processing algorithm finds specific details in the user’s eyes and reflection mode, based on this information the eye’s position and gaze point are calculated.

In addition, we chose a 15.6-in display notebook to connect the eye tracker, and the resolution of the notebook is 1920 × 1080. The Tobii Eye Tracker 4C only needs to be fixed under the experimental computer screen, which is a good solution to reduce the experimental error caused by the uncomfortable side effects of long-time experiment.

Clear View data analysis software was used to analyze eye movement data, the interface, and the video of the user’s action.

Experiment Procedure

The participants were asked to complete a profile questionnaire (demographic information) and an informed consent form first. Then, experiment leader explained the experimental purpose and procedure to the participants. Then the formal experiment started. First, eye movement calibration was conducted. The participants were told to sit upright in the chair with their eyes facing the front, and to try to maintain a stable posture. The experimenter calibrated the eye gaze for the participant. During the calibration process, the participants were reminded that there would be three groups of dots in different corners on the screen. The calibration interface is as shown in Figure 2A. After the calibration, we tested the calibration effect. By looking at the calibration points in the circle, we can determine whether the calibration was accurate enough or not. If the gaze point was in the circle, the calibration was considered good enough; otherwise, we would perform the calibration once again. The test interface is as shown in Figure 2B.

FIGURE 2

Figure 2. Eye tracker calibration and test interface. (A) Calibration interface. (B) Test interface.

Next, a set of target squares (one of five positions) and a circular target (around the target square) would be randomly displayed on the screen. The participant was asked to use eye movement to drag the target to the specified circular range, and then to drop the target. Participants were required to complete two sets of experiments with two clicks (blink and dwell). After a set of a target square and a circular target was displayed on the screen, the participants fixated on the target square for one second to pick up the target square. Then, he/she needed to drag the target square to the circular range and put it down by fixating for one second. This was a complete operation. Participants could drag and drop multiple times until the target square was successfully placed in the circular range. Three seconds after the completion of one operation, the target would randomly appear in the next position. The operation in five locations is one group of experiments, and the experiment was repeated for four groups, i.e., 20 trials per participant in total for each target clicking method. After the dwell experiment was completed, the target clicking method changed to blink, and the experimental procedure remained the same.

Data Analysis

We performed a repeated measures analysis of variances (RMANOVA) and categorical regression with optimal scaling (CATREG) on the dependent variables. Categorical regression quantifies categorical data by assigning numerical values to the categories, resulting in an optimal linear regression equation for the transformed variables. Multivariable analysis that included Pillai’s trace, Wilks’ lambda, Hotelling’s trace, and Roy’s largest root was used to test the effect of the increase in experiment time and operation repetition number on the experimental results. Then, according to the result of Mauchly’s test of sphericity (p < 0.05), we analyzed the corrected part of the unary analysis for studying the effect of target layout and target clicking method on picking time and dragging performance.

Results

Picking Time

The results of repeated measures analysis of the picking time are shown in Table 1. Multivariable analysis that included Pillai’s trace, Wilks’ lambda, Hotelling’s trace, and Roy’s largest root was used to test the effect of the increase in experiment time and operation repetition number on the experimental results. The result (p > 0.05) showed that the experiment time and operation repetition number did not make a difference in the five values of the target’s initial position and the two target clicking methods, which indicated that the increase in experiment time and operation repetition number did not affect the experimental results (picking time).

TABLE 1

Table 1. The repeated measures analysis.

According to the result of Mauchly’s sphericity test (p < 0.05, see Table 2), we should analyze the results of the corrected tests (see Table 3). The results indicated that the position of the target (p = 0.083) and the target clicking method (p = 0.295) had no significant effect on the picking time.

TABLE 2

Table 2. Mauchly’s sphericity test.

TABLE 3

Table 3. Tests of within subjects.

From the results of the ANOVAs, it can be seen that there was no significant difference in the picking time with different target picking methods and initial positions of the target, so the regression analysis would be meaningless for picking time.

Dragging Time

The results of a repeated measures analysis of the dragging time are shown in Table 1. The results (p > 0.05) indicated that increases in experiment time and operation repetition number did not affect the experimental results (dragging time).

According to the result of Mauchly’s sphericity test (p < 0.05, see Table 2), we should analyze the results of corrected tests (see Table 3). The results indicated that the target clicking method (p = 0.013) had a significant effect on the dragging time. However, the position of the target (p = 0.211) and the target clicking method × the position of the target (p = 0.438) had no significant effect on the dragging time.

Then, the regression analysis was discussed. It can be seen from Table 4 that the overall regression model (p = 0.016 < 0.05) has statistical significance, i.e., overall, the regression model statistically significantly predicts the outcome variable. Table 5 shows the standardized coefficients and the test results of each coefficient in the model. The results indicated that the target clicking method (p = 0.001 < 0.05) had a significant effect on the dragging time and that the position of the target (p = 0.739) had no significant effect on the dragging time. Focusing on Table 6, the result (Importance) shows that a strong predictor is the target clicking method and that the position of the target is not a significant predictor of dragging time.

TABLE 4

Table 4. The ANOVA of the regression model.

TABLE 5

Table 5. Coefficients of regression models.

TABLE 6

Table 6. Correlations and tolerance of the dragging time.

Dragging Numbers

The results of a repeated measures analysis of the dragging numbers are shown in Table 1. The result (p > 0.05) indicated that the increase in experiment time and operation repetition number did not affect the experimental results (dragging numbers).

According to the result of Mauchly’s sphericity test (p < 0.05, see Table 2), we should analyze the results of corrected tests (see Table 3). The results indicated that the target clicking method (p = 0.013) had a significant effect on the dragging numbers. However, the position of the target (p = 0.211) and the target clicking method × the position of the target (p = 0.438) had no significant effect on the dragging numbers.

The target clicking method had a significant effect on the dragging numbers; therefore, regression analysis was carried out. Table 4 shows that the overall regression model (p = 0.066) has no statistical significance. However, we can see the effect of each of the predictors on the dragging numbers in Table 5. The result shows that the target clicking method (p = 0.001 < 0.05) had a significant effect on the dragging numbers and that the position of the target (p = 0.828) had no significant effect on the dragging numbers. Table 6 shows that a strong predictor is the target clicking method and that the position of the target is not a significant predictor of dragging numbers.

Target Picking Method

Through the analyses of the results, it can be concluded that the target picking method has significant effects on the dragging time and the dragging numbers. A comparison of the two target picking methods is shown in Figure 3. When the target picking method is dwell, the manipulation for the interface is better, the time of the complete dragging command is shorter, and the error rate is lower.

FIGURE 3

Figure 3. Comparison of the two target picking methods.

Discussion

The results show that the target layout has no significant effect on the picking time and drag performance (dragging time and dragging numbers), which is inconsistent with our previous assumptions, so Hypothesis 2 cannot be accepted at the significance level of 0.05, which may be related to the set experimental conditions (such as light level and screen resolution). However, the target clicking method (blinking and dwell) has a significant effect on picking dragging time and the dragging number. The experimental results agree with Hypothesis 3. The target clicking method and target layout have no significant interaction impact on picking time and drag performance, which is consistent with Hypothesis 4.

According to the results of CATREG, the overall regression model has statistical significance. The relationships between the target clicking method and the dragging time and numbers are statistically significant. However, the position of the target had no significant effect on the dragging time and numbers. The effect of the target clicking method on dragging performance is stronger than the effect of the initial position of the target.

Conclusion

The study considers the influence of experiment time and operation repetition number on the experiment performance, and evaluates the impact of target layout and clicking method on picking time and drag performance from the perspective of efficiency and effectiveness. The experimental results show that the target layout has no significant effect on the picking time and drag performance (dragging time and drag numbers). However, the target click method, i.e., blink and dwell, has a significant impact on the dragging time and numbers, and compared to using blink, using dwell to click can identify the target better, with shorter time and higher accuracy. The findings are anticipated to provide helpful implications for future eye control technique design and HCI interface design.

Data Availability Statement

The datasets analyzed in this article are not publicly available because of privacy policies. Requests to access the datasets should be directed to JN, niujw@ustb.edu.cn.

Ethics Statement

The studies involving human participants were reviewed and approved by the University of Science and Technology Beijing, Beijing, China. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

LW and DW contributed to validation, formal analysis, writing—original draft preparation, and writing—review and editing. YZ contributed to conceptualization, methodology, project administration, and funding acquisition. HL contributed to methodology, software, and visualization. JS contributed to methodology and formal analysis. YZ contributed to methodology and validation. CZ contributed to conceptualization, methodology, and formal analysis. JN contributed to conceptualization, methodology, formal analysis, writing—review and editing, and funding acquisition. All authors contributed to the article and approved the submitted version.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

This article is an extended version of our previous conference paper “The influence of target layout on dragging performance based on eye-control technique” published in Proceedings of the Second International Conference on Intelligent Human Systems Integration (IHSI 2019): Integrating People and Intelligent Systems, 7–10 February 2019, San Diego, CA, United States.

References

Duchowski, A. (2007). Eye Tracking Methodology: Theory And Practice. London: Springer.

Google Scholar

Fitts, P. M., Jones, R. E., and Milton, J. L. (1949). Eye fixations of aircraft pilots. III. Frequency, duration and sequence fixations when flying air force ground-controlled approach system (GCA). Air Mater. Comm. 10, 1–25.

Google Scholar

Fitts, P. M., Jones, R. E., and Milton, J. L. (1950). Eye movements of aircraft pilots during instrument landing approaches. Aeronaut. Eng. Rev. 9, 1–5.

Google Scholar

Goldberg, J. H., and Helfman, J. I. (2011). Eye tracking for visualization evaluation: reading values on linear versus radial graphs. Inform. Vis. 10, 182–195. doi: 10.1177/1473871611406623

CrossRef Full Text | Google Scholar

Harezlak, K., and Kasprowski, P. (2018). Application of eye tracking in medicine: a survey, research issues and challenges. Comput. Med. Imaging Graph. 65, 176–190. doi: 10.1016/j.compmedimag.2017.04.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Jacob, R. J., and Karn, S. K. (2003). “Eye tracking in human-computer interaction and usability research: ready to deliver the promises,” in The Mind’s Eye: Cognitive and Applied Aspects Of Eye Movement Research, eds J. Hyönä, R. Radach, and H. Deubel (Amsterdam: Elsevier Science), 573–605. doi: 10.1016/b978-044451020-4/50031-1

CrossRef Full Text | Google Scholar

Lin, C. S., Huan, C. C., Chan, C. N., Yeh, M. S., and Chiu, C. C. (2004). Design of a computer game using an eye-tracking device for eye’s activity rehabilitation. Opt. Lasers Eng. 42, 91–108. doi: 10.1016/s0143-8166(03)00075-7

CrossRef Full Text | Google Scholar

Liu, L. L., He, X. Z., and Fu, C. W. (2016). “Study on usability of agricultural product web page layout based on eye tracker,” in Proceedings of the International Conference on Automatic Control and Information Engineering, Paris.

Google Scholar

Liversedge, S., Gilchrist, I., and Everling, S. (2011). The Oxford Handbook Of Eye Movements. Oxford: Oxford University Press.

Google Scholar

Muehlethaler, C. M., and Knecht, C. P. (2016). Situation awareness training for general aviation pilots using eye tracking. IFAC Pap. 49, 66–71. doi: 10.1016/j.ifacol.2016.10.463

CrossRef Full Text | Google Scholar

Pušnik, N., Tihole, K., and Možina, K. (2016). “Testing magazine design with eye-tracking technology,” in Proceedings of the the 8th International Symposium on Graphic Engineering and Design, Novi Sad.

Google Scholar

Rayner, K. (2009). Eye movements and attention in reading, scene perception, and visual search. Q. J. Exp. Psychol. 62, 1457–1506. doi: 10.1080/17470210902816461

PubMed Abstract | CrossRef Full Text | Google Scholar

Raynor, K., and Pollatsek, A. (1994). The Psychology of Reading. New Jersey: Lawrence Erlbaum Associates.

Google Scholar

Renshaw, J. A., Finlay, J. E., Ward, R. D., and Tyfa, D. (2003). “Designing for visual influence: an eye tracking study of the usability of graphical management information,” in Proceedings of the IFIP Conference On Human-Computer Interaction, London.

Google Scholar

Underwood, G., and Radach, R. (1998). “Eye guidance and visual information processing: reading, visual search, picture perception and driving,” in Eye Guidance In Reading And Scene Perception, ed. G. Underwood (Oxford: Elsevier), 1–28.

Google Scholar

Wu, C. (2012). HCI and eye tracking technology for learning effect. Proc. Soc. Behav. Sci. 64, 626–632. doi: 10.1016/j.sbspro.2012.11.073

CrossRef Full Text | Google Scholar

Zhao, S. Q., Xiang, C. L., Su, L., Jiang, Z. Y., and Liao, C. J. (2018). Study on the eye movement characteristics of fire hazard identification in university laboratories. Proc. Eng. 211, 433–440. doi: 10.1016/j.proeng.2017.12.033

CrossRef Full Text | Google Scholar