应用眼动追踪和机器学习评估学生在 iVR 学习环境中表现的方法

 Ana Serrano-Mamolar, Burgos (西班牙)

 Ines Miguel-Alonso, Burgos (西班牙)

 David Checa, Burgos (西班牙)

 Carlos Pardo-Aguilar, Burgos (西班牙)

https://doi.org/10.3916/C76-2023-01

关键词

虚拟环境、基于游戏的学习、机器学习、眼动追踪、特征提取、神经教育

摘要

目前，由于眼动追踪技术的低侵入性及其在商业虚拟现实眼镜设备中的集成，在沉浸式虚拟现实 (iVR) 学习环境中使用眼动追踪数据会成为将学习成果最大化的有力工具。但是，在学习环境中使用眼动追踪之前，必须确定最适合数据处理的技术。本研究建议为该目的使用机器学习技术，评估它们对学习环境质量进行分类和预测用户学习表现的能力。为此，我们在 iVR 中开发了一种教学体验，用于学习高架起重机的操作。通过这次体验，我们对 63 名学生的表现进行了评估，无论是在最佳学习条件下还是在有压力的条件下。最终的数据集包括 25 个特征，大部分是时间序列，数据集大小大于 5000 万个点。结果表明，KNN、SVM 或随机森林等不同分类器的应用在预测学习变化时表现了较高的精度，而对用户学习性能的预测仍远未得到优化。这一点为未来的研究开辟了新的思路。本研究旨在通过使用更复杂的机器学习技术作为未来改进模型准确性的基线。

参考资料

Añaños-Carrasco, E. (2015). Eyetracker technology in elderly people: How integrated television content is paid attention to and processed. [La tecnología del «EyeTracker» en adultos mayores: Cómo se atienden y procesan los contenidos integrados de televisión]. Comunicar, 45, 75-83. https://doi.org/10.3916/C45-2015-08

Link DOI | Link Google Scholar

Asish, S.M., Kulshreshth, A.K., & Borst, C.W. (2022). Detecting distracted students in educational VR environments using machine learning on eye gaze data. Computers & Graphics, 109, 75-87. https://doi.org/10.1016/j.cag.2022.10.007

Link DOI | Link Google Scholar

Bowman, D.A., & McMahan, R.P. (2007). Virtual reality: How much immersion is enough? Computer, 40(7), 36-43. https://doi.org/10.1109/MC.2007.257

Link DOI | Link Google Scholar

Checa, D., & Bustillo, A. (2020). A review of immersive virtual reality serious games to enhance learning and training. Multimedia Tools and Applications, 79(9-10), 5501–5527. https://doi.org/10.1007/s11042-019-08348-9

Link DOI | Link Google Scholar

Checa, D., & Bustillo, A. (2022). Grua Rv. http://3dubu.Es/En/Cranevr/

Link Google Scholar

Checa, D., Gatto, C., Cisternino, D., de Paolis, L.T., & Bustillo, A. (2020). A Framework for Educational and Training Immersive Virtual Reality Experiences. In L.T. de-Paolis & P. Bourdot (Eds.), Augmented reality, virtual reality, and computer graphics (pp. 220–228). Springer International Publishing. https://doi.org/10.1007/978-3-030-58468-9_17

Link DOI | Link Google Scholar

Christ, M., Braun, N., Neuffer, J., & Kempa-Liehr, A.W. (2018). Time series feature extraction on basis of scalable hypothesis tests (tsfresh – A Python package). Neurocomputing, 307, 72-77. https://doi.org/https://doi.org/10.1016/j.neucom.2018.03.067

Link Google Scholar

Christ, M., Kempa-Liehr, A., & Feindt, M. (2016). Distributed and parallel time series feature extraction for industrial big data applications. ArXiv, 1, https://doi.org/10.48550/arXiv.1610.07717.

Link DOI | Link Google Scholar

Cowan, A., Chen, J., Mingo, S., Reddy, S.S., Ma, R., Marshall, S., Nguyen, J.H., & Hung, A.J. (2021). virtual reality vs dry laboratory models: Comparing automated performance metrics and cognitive workload during robotic simulation training. Journal of Endourology, 35(10), 1571-1576. https://doi.org/10.1089/end.2020.1037

Link DOI | Link Google Scholar

Dale, E. (1946). Audiovisual methods in teaching. Dryden Press. https://bit.ly/42aW03X

Link Google Scholar

Dalgarno, B., & Lee, M.J.W. (2010). What are the learning affordances of 3-D virtual environments? British Journal of Educational Technology, 41(1). https://doi.org/10.1111/j.1467-8535.2009.01038.x

Link DOI | Link Google Scholar

Dede, C. (2009). Immersive interfaces for engagement and learning. Science, 323(5910), 66-69. https://doi.org/10.1126/science.1167311

Link DOI | Link Google Scholar

Deng, Q., Wang, J., Hillebrand, K., Benjamin, C.R., & Soffker, D. (2020). Prediction performance of lane changing behaviors: A study of combining environmental and eye-tracking data in a driving simulator. IEEE Transactions on Intelligent Transportation Systems, 21(8), 3561-3570. https://doi.org/10.1109/TITS.2019.2937287

Link DOI | Link Google Scholar

Duchowski, A. T. (2002). A breadth-first survey of eye-tracking applications. Behavior Research Methods, Instruments, & Computers, 34(4), 455-470. https://doi.org/10.3758/BF03195475

Link DOI | Link Google Scholar

Farran, E., Formby, S., Daniyal, F., Holmes, T., & Herwegen, J. (2016). Route-learning strategies in typical and atypical development; eye-tracking reveals atypical landmark selection in Williams syndrome: Route-learning and eye-tracking. Journal of Intellectual Disability Research, 60(10), 933-944. https://doi.org/10.1111/jir.12331

Link DOI | Link Google Scholar

García-Carrasco, J., Hernández-Serrano, M.J., & Martín-García, A.V. (2015). Plasticity as a framing concept enabling transdisciplinary understanding and research in neuroscience and education. Learning, Media and Technology, 40(2), 152-167. https://doi.org/10.1080/17439884.2014.908907

Link DOI | Link Google Scholar

Gardony, A.L., Lindeman, R.W., & Brunyé, T.T. (2020). Eye-tracking for human-centered mixed reality: Promises and challenges. Proc.SPIE, 11310, 113100T. https://doi.org/10.1117/12.2542699

Link DOI | Link Google Scholar

Glennon, J.M., D’Souza, H., Mason, L., Karmiloff-Smith, A., & Thomas, M.S.C. (2020). Visuo-attentional correlates of Autism Spectrum Disorder (ASD) in children with Down syndrome: A comparative study with children with idiopathic ASD. Research in Developmental Disabilities, 104, 103678. https://doi.org/10.1016/j.ridd.2020.103678

Link DOI | Link Google Scholar

Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. (2008). The WEKA data mining software: An update. SIGKDD Explor. Newsl., 11(1), 10-18. https://doi.org/10.1145/1656274.1656278

Link DOI | Link Google Scholar

Huang, H.M., Rauch, U., & Liaw, S.S. (2010). Investigating learners’ attitudes toward virtual reality learning environments: Based on a constructivist approach. Computers and Education, 55(3), 1171-1182. https://doi.org/10.1016/j.compedu.2010.05.014

Link DOI | Link Google Scholar

Lapborisuth, P., Koorathota, S., Wang, Q., & Sajda, P. (2021). Integrating neural and ocular attention reorienting signals in virtual reality. Journal of Neural Engineering, 18(6), 066052. https://doi.org/10.1088/1741-2552/ac4593

Link DOI | Link Google Scholar

Ma, X., Yao, Z., Wang, Y., Pei, W., & Chen, H. (2018). Combining brain-computer interface and eye-tracking for high-speed text entry in virtual reality. In Berkovsky & Y. Hijikata (Ed.), IUI ’18: 23rd International Conference on Intelligent User Interfaces (pp. 263-267). https://doi.org/10.1145/3172944.3172988

Link DOI | Link Google Scholar

Martinez, K., Menéndez-Menéndez, M.I., & Bustillo, A. (2021). Awareness, prevention, detection, and therapy applications for depression and anxiety in serious games for children and adolescents: Systematic review. JMIR Serious Games, 9(4), e30482. https://doi.org/10.2196/30482

Link DOI | Link Google Scholar

Mckinney, W. (2011). pandas: A foundational Python library for data analysis and statistics. Python High Performance Science Computer.

Link Google Scholar

Patney, A., Salvi, M., Kim, J., Kaplanyan, A., Wyman, C., Benty, N., Luebke, D., & Lefohn, A. (2016). Towards foveated rendering for gaze-tracked virtual reality. ACM Trans. Graph., 35(6), 1-12. https://doi.org/10.1145/2980179.2980246

Link DOI | Link Google Scholar

Pritchard, A. (2017). Ways of learning: Learning theories for the classroom. Routledge. https://doi.org/10.4324/9781315460611

Link DOI | Link Google Scholar

Rappa, N.A., Ledger, S., Teo, T., Wai Wong, K., Power, B., & Hilliard, B. (2022). The use of eye-tracking technology to explore learning and performance within virtual reality and mixed reality settings: A scoping review. Interactive Learning Environments, 30(7), 1338-1350. https://doi.org/10.1080/10494820.2019.1702560

Link DOI | Link Google Scholar

Rodero, E., & Larrea, O. (2022). Virtual reality with distractors to overcome public speaking anxiety in university students; [Realidad virtual con distractores para superar el miedo a hablar en público en universitarios]. Comunicar, 30(72). https://doi.org/10.3916/C72-2022-07

Link DOI | Link Google Scholar

Shadiev, R., & Li, D. (2022). A review study on eye-tracking technology usage in immersive virtual reality learning environments. Computers & Education, 104681. https://doi.org/10.1016/j.compedu.2022.104681

Link DOI | Link Google Scholar

Sun, Q., Patney, A., Wei, L.Y., Shapira, O., Lu, J., Asente, P., Zhu, S., McGuire, M., Luebke, D., & Kaufman, A. (2018). Towards virtual reality infinite walking: Dynamic saccadic redirection. ACM Transactions on Graphics, 37(4), 1-13. https://doi.org/10.1145/3197517.3201294

Link DOI | Link Google Scholar

Tanaka, Y., Kanari, K., & Sato, M. (2021). Interaction with virtual objects through eye-tracking. In M. Nakajima, J.G. Kim, W.N. Lie, & Q. Kemao (Eds.), International Workshop on Advanced Imaging Technology (IWAIT) 2021 (p. 1176624). SPIE. https://doi.org/10.1117/12.2590989

Link DOI | Link Google Scholar

Tavenard, R., Faouzi, J., Vandewiele, G., Divo, F., Androz, G., Holtz, C., Payne, M., Yurchak, R., Rußwurm, M., Kolar, K., & Woods, E. (2020). Tslearn, A Machine-learning Toolkit for Time Series Data. J. Mach. Learn. Res., 21, 118, 1-6.

Link Google Scholar

Wilson, M. (2002). Six views of embodied cognition. Psychonomic Bulletin and Review, 9, 625-636. https://doi.org/10.3758/BF03196322

Link DOI | Link Google Scholar

Wismer, P., Soares, S.A., Einarson, K.A., & Sommer, M.O.A. (2022). Laboratory performance prediction using virtual reality behaviometrics. PloS One, 17(12), e0279320. https://doi.org/10.1371/journal.pone.0279320

Link DOI | Link Google Scholar

Fundref

This work was partially supported by the ACIS Project (Reference Number: INVESTUN/21/BU/0002) of the Consejeria de Empleo e Industria of the Junta de Castilla y León (Spain), the Erasmus+ RISKREAL Project (2020-1-ES01-KA204-081847) of the European Commission, the HumanAid Project (TED2021-129485B-C43) of the Spanish Ministry of Science and Innovation, and the Margarita Salas program of the Spanish Ministry of Universities funded by NextGenerationEU.

Crossmark

技术信息

收到: 26-12-2022

修订: 25-01-2023

公认: 23-02-2023

OnlineFirst: 30-05-2023

发布日期: 01-07-2023

文章修改时间: 30 天 | 期刊编号的平均时间修订 76: -6 天

文章接受时间: 59 天 | 期刊编号的平均接受时间 76: 72 天

预印本编辑时间: 141 天 | 期刊编号的平均编辑时间预印 76: 154 天

文章编辑时间: 186 天 | 期刊编号的平均编辑时间 76: 199 天

度量

这篇文章的度量

浏览次数: 34361

对摘要的解读: 33257

PDF下载: 1104

的全部指标 76

浏览次数: 386179

对摘要的解读: 377280

PDF下载: 8899

被引用