Volume index - Journal index - Article index - Map ---- Back

Comunicar Journal 76: Neurotechnology in the classroom: Current research and future potential (Vol. 31 - 2023)

Towards learner performance evaluation in iVR learning environments using eye-tracking and Machine-learning

https://doi.org/10.3916/C76-2023-01

Ana Serrano-Mamolar

Ines Miguel-Alonso

David Checa

Carlos Pardo-Aguilar

Abstract

At present, the use of eye-tracking data in immersive Virtual Reality (iVR) learning environments is set to become a powerful tool for maximizing learning outcomes, due to the low-intrusiveness of eye-tracking technology and its integration in commercial iVR Head Mounted Displays. However, the most suitable technologies for data processing should first be identified before their use in learning environments can be generalized. In this research, the use of machine-learning techniques is proposed for that purpose, evaluating their capabilities to classify the quality of the learning environment and to predict user learning performance. To do so, an iVR learning experience simulating the operation of a bridge crane was developed. Through this experience, the performance of 63 students was evaluated, both under optimum learning conditions and under stressful conditions. The final dataset included 25 features, mostly temporal series, with a dataset size of up to 50M data points. The results showed that different classifiers (KNN, SVM and Random Forest) provided the highest accuracy when predicting learning performance variations, while the accuracy of user learning performance was still far from optimized, opening a new line of future research. This study has the objective of serving as a baseline for future improvements to model accuracy using complex machine-learning techniques.

Keywords

Virtual environment, game-based learning, machinelearning, eye-tracking, feature extraction, neuroeducation

PDF file in Spanish

PDF file in English

References

Añaños-Carrasco, E. (2015). Eyetracker technology in elderly people: How integrated television content is paid attention to and processed. [La tecnología del «EyeTracker» en adultos mayores: Cómo se atienden y procesan los contenidos integrados de televisión]. Comunicar, 45, 75-83. https://doi.org/10.3916/C45-2015-08

Asish, S.M., Kulshreshth, A.K., & Borst, C.W. (2022). Detecting distracted students in educational VR environments using machine learning on eye gaze data. Computers & Graphics, 109, 75-87. https://doi.org/10.1016/j.cag.2022.10.007

Bowman, D.A., & McMahan, R.P. (2007). Virtual reality: How much immersion is enough? Computer, 40(7), 36-43. https://doi.org/10.1109/MC.2007.257

Checa, D., & Bustillo, A. (2020). A review of immersive virtual reality serious games to enhance learning and training. Multimedia Tools and Applications, 79(9-10), 5501–5527. https://doi.org/10.1007/s11042-019-08348-9

Checa, D., & Bustillo, A. (2022). Grua Rv. http://3dubu.Es/En/Cranevr/

Checa, D., Gatto, C., Cisternino, D., de Paolis, L.T., & Bustillo, A. (2020). A Framework for Educational and Training Immersive Virtual Reality Experiences. In L.T. de-Paolis & P. Bourdot (Eds.), Augmented reality, virtual reality, and computer graphics (pp. 220–228). Springer International Publishing. https://doi.org/10.1007/978-3-030-58468-9_17

Christ, M., Braun, N., Neuffer, J., & Kempa-Liehr, A.W. (2018). Time series feature extraction on basis of scalable hypothesis tests (tsfresh – A Python package). Neurocomputing, 307, 72-77. https://doi.org/https://doi.org/10.1016/j.neucom.2018.03.067

Christ, M., Kempa-Liehr, A., & Feindt, M. (2016). Distributed and parallel time series feature extraction for industrial big data applications. ArXiv, 1, https://doi.org/10.48550/arXiv.1610.07717.

Cowan, A., Chen, J., Mingo, S., Reddy, S.S., Ma, R., Marshall, S., Nguyen, J.H., & Hung, A.J. (2021). virtual reality vs dry laboratory models: Comparing automated performance metrics and cognitive workload during robotic simulation training. Journal of Endourology, 35(10), 1571-1576. https://doi.org/10.1089/end.2020.1037

Dale, E. (1946). Audiovisual methods in teaching. Dryden Press. https://bit.ly/42aW03X

Dalgarno, B., & Lee, M.J.W. (2010). What are the learning affordances of 3-D virtual environments? British Journal of Educational Technology, 41(1). https://doi.org/10.1111/j.1467-8535.2009.01038.x

Dede, C. (2009). Immersive interfaces for engagement and learning. Science, 323(5910), 66-69. https://doi.org/10.1126/science.1167311

Deng, Q., Wang, J., Hillebrand, K., Benjamin, C.R., & Soffker, D. (2020). Prediction performance of lane changing behaviors: A study of combining environmental and eye-tracking data in a driving simulator. IEEE Transactions on Intelligent Transportation Systems, 21(8), 3561-3570. https://doi.org/10.1109/TITS.2019.2937287

Duchowski, A. T. (2002). A breadth-first survey of eye-tracking applications. Behavior Research Methods, Instruments, & Computers, 34(4), 455-470. https://doi.org/10.3758/BF03195475

Farran, E., Formby, S., Daniyal, F., Holmes, T., & Herwegen, J. (2016). Route-learning strategies in typical and atypical development; eye-tracking reveals atypical landmark selection in Williams syndrome: Route-learning and eye-tracking. Journal of Intellectual Disability Research, 60(10), 933-944. https://doi.org/10.1111/jir.12331

García-Carrasco, J., Hernández-Serrano, M.J., & Martín-García, A.V. (2015). Plasticity as a framing concept enabling transdisciplinary understanding and research in neuroscience and education. Learning, Media and Technology, 40(2), 152-167. https://doi.org/10.1080/17439884.2014.908907

Gardony, A.L., Lindeman, R.W., & Brunyé, T.T. (2020). Eye-tracking for human-centered mixed reality: Promises and challenges. Proc.SPIE, 11310, 113100T. https://doi.org/10.1117/12.2542699

Glennon, J.M., D’Souza, H., Mason, L., Karmiloff-Smith, A., & Thomas, M.S.C. (2020). Visuo-attentional correlates of Autism Spectrum Disorder (ASD) in children with Down syndrome: A comparative study with children with idiopathic ASD. Research in Developmental Disabilities, 104, 103678. https://doi.org/10.1016/j.ridd.2020.103678

Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. (2008). The WEKA data mining software: An update. SIGKDD Explor. Newsl., 11(1), 10-18. https://doi.org/10.1145/1656274.1656278

Huang, H.M., Rauch, U., & Liaw, S.S. (2010). Investigating learners’ attitudes toward virtual reality learning environments: Based on a constructivist approach. Computers and Education, 55(3), 1171-1182. https://doi.org/10.1016/j.compedu.2010.05.014

Lapborisuth, P., Koorathota, S., Wang, Q., & Sajda, P. (2021). Integrating neural and ocular attention reorienting signals in virtual reality. Journal of Neural Engineering, 18(6), 066052. https://doi.org/10.1088/1741-2552/ac4593

Ma, X., Yao, Z., Wang, Y., Pei, W., & Chen, H. (2018). Combining brain-computer interface and eye-tracking for high-speed text entry in virtual reality. In Berkovsky & Y. Hijikata (Ed.), IUI ’18: 23rd International Conference on Intelligent User Interfaces (pp. 263-267). https://doi.org/10.1145/3172944.3172988

Martinez, K., Menéndez-Menéndez, M.I., & Bustillo, A. (2021). Awareness, prevention, detection, and therapy applications for depression and anxiety in serious games for children and adolescents: Systematic review. JMIR Serious Games, 9(4), e30482. https://doi.org/10.2196/30482

Mckinney, W. (2011). pandas: A foundational Python library for data analysis and statistics. Python High Performance Science Computer.

Patney, A., Salvi, M., Kim, J., Kaplanyan, A., Wyman, C., Benty, N., Luebke, D., & Lefohn, A. (2016). Towards foveated rendering for gaze-tracked virtual reality. ACM Trans. Graph., 35(6), 1-12. https://doi.org/10.1145/2980179.2980246

Pritchard, A. (2017). Ways of learning: Learning theories for the classroom. Routledge. https://doi.org/10.4324/9781315460611

Rappa, N.A., Ledger, S., Teo, T., Wai Wong, K., Power, B., & Hilliard, B. (2022). The use of eye-tracking technology to explore learning and performance within virtual reality and mixed reality settings: A scoping review. Interactive Learning Environments, 30(7), 1338-1350. https://doi.org/10.1080/10494820.2019.1702560

Rodero, E., & Larrea, O. (2022). Virtual reality with distractors to overcome public speaking anxiety in university students; [Realidad virtual con distractores para superar el miedo a hablar en público en universitarios]. Comunicar, 30(72). https://doi.org/10.3916/C72-2022-07

Shadiev, R., & Li, D. (2022). A review study on eye-tracking technology usage in immersive virtual reality learning environments. Computers & Education, 104681. https://doi.org/10.1016/j.compedu.2022.104681

Sun, Q., Patney, A., Wei, L.Y., Shapira, O., Lu, J., Asente, P., Zhu, S., McGuire, M., Luebke, D., & Kaufman, A. (2018). Towards virtual reality infinite walking: Dynamic saccadic redirection. ACM Transactions on Graphics, 37(4), 1-13. https://doi.org/10.1145/3197517.3201294

Tanaka, Y., Kanari, K., & Sato, M. (2021). Interaction with virtual objects through eye-tracking. In M. Nakajima, J.G. Kim, W.N. Lie, & Q. Kemao (Eds.), International Workshop on Advanced Imaging Technology (IWAIT) 2021 (p. 1176624). SPIE. https://doi.org/10.1117/12.2590989

Tavenard, R., Faouzi, J., Vandewiele, G., Divo, F., Androz, G., Holtz, C., Payne, M., Yurchak, R., Rußwurm, M., Kolar, K., & Woods, E. (2020). Tslearn, A Machine-learning Toolkit for Time Series Data. J. Mach. Learn. Res., 21, 118, 1-6.

Wilson, M. (2002). Six views of embodied cognition. Psychonomic Bulletin and Review, 9, 625-636. https://doi.org/10.3758/BF03196322

Wismer, P., Soares, S.A., Einarson, K.A., & Sommer, M.O.A. (2022). Laboratory performance prediction using virtual reality behaviometrics. PloS One, 17(12), e0279320. https://doi.org/10.1371/journal.pone.0279320