Palavras chave
Ambiente virtual, aprendizado baseado em jogos, aprendizado de máquina, registro do olhar, extração de características, neuroeducação
Resumo
Atualmente, o uso de dados de rastreamento ocular em ambientes imersivos de aprendizagem de Realidade Virtual (iAVR) está destinado a ser uma ferramenta fundamental para maximizar os resultados de aprendizagem, dada a natureza não intrusiva do rastreamento ocular e sua integração em óculos comerciais de realidade virtual. Mas, antes que o uso de rastreamento ocular em ambientes de aprendizagem possa ser generalizado, as tecnologias mais adequadas para processamento de dados devem ser identificadas. Esta pesquisa propõe o uso de técnicas de aprendizado de máquina para esse fim, avaliando suas habilidades para classificar a qualidade do ambiente de aprendizagem e prever o desempenho de aprendizado do usuário. Para isso, foi desenvolvida uma experiência de ensino em iAVR para aprender o manuseio de uma ponte rolante. Com esta experiência, avaliou-se o desempenho de 63 alunos, tanto em condições ótimas de aprendizagem como em condições com estressores. O conjunto de dados final inclui 25 feições, sendo a maioria composta por séries temporais com um tamanho de conjunto de dados superior a 50 milhões de pontos. Os resultados mostram que a aplicação de diferentes classificações como KNN, SVM ou Random Forest tem alta precisão na previsão de alterações de aprendizagem, enquanto a previsão do desempenho de aprendizagem do usuário ainda está longe do ideal, o que abre uma linha de pesquisa futura. Este estudo visa servir como base para melhorias futuras na precisão do modelo, usando técnicas de aprendizado de máquina mais complexas.
Referências
Añaños-Carrasco, E. (2015). Eyetracker technology in elderly people: How integrated television content is paid attention to and processed. [La tecnología del «EyeTracker» en adultos mayores: Cómo se atienden y procesan los contenidos integrados de televisión]. Comunicar, 45, 75-83. https://doi.org/10.3916/C45-2015-08
Link DOI | Link Google Scholar
Asish, S.M., Kulshreshth, A.K., & Borst, C.W. (2022). Detecting distracted students in educational VR environments using machine learning on eye gaze data. Computers & Graphics, 109, 75-87. https://doi.org/10.1016/j.cag.2022.10.007
Link DOI | Link Google Scholar
Bowman, D.A., & McMahan, R.P. (2007). Virtual reality: How much immersion is enough? Computer, 40(7), 36-43. https://doi.org/10.1109/MC.2007.257
Link DOI | Link Google Scholar
Checa, D., & Bustillo, A. (2020). A review of immersive virtual reality serious games to enhance learning and training. Multimedia Tools and Applications, 79(9-10), 5501–5527. https://doi.org/10.1007/s11042-019-08348-9
Link DOI | Link Google Scholar
Checa, D., & Bustillo, A. (2022). Grua Rv. http://3dubu.Es/En/Cranevr/
Link Google Scholar
Checa, D., Gatto, C., Cisternino, D., de Paolis, L.T., & Bustillo, A. (2020). A Framework for Educational and Training Immersive Virtual Reality Experiences. In L.T. de-Paolis & P. Bourdot (Eds.), Augmented reality, virtual reality, and computer graphics (pp. 220–228). Springer International Publishing. https://doi.org/10.1007/978-3-030-58468-9_17
Link DOI | Link Google Scholar
Christ, M., Braun, N., Neuffer, J., & Kempa-Liehr, A.W. (2018). Time series feature extraction on basis of scalable hypothesis tests (tsfresh – A Python package). Neurocomputing, 307, 72-77. https://doi.org/https://doi.org/10.1016/j.neucom.2018.03.067
Link Google Scholar
Christ, M., Kempa-Liehr, A., & Feindt, M. (2016). Distributed and parallel time series feature extraction for industrial big data applications. ArXiv, 1, https://doi.org/10.48550/arXiv.1610.07717.
Link DOI | Link Google Scholar
Cowan, A., Chen, J., Mingo, S., Reddy, S.S., Ma, R., Marshall, S., Nguyen, J.H., & Hung, A.J. (2021). virtual reality vs dry laboratory models: Comparing automated performance metrics and cognitive workload during robotic simulation training. Journal of Endourology, 35(10), 1571-1576. https://doi.org/10.1089/end.2020.1037
Link DOI | Link Google Scholar
Dale, E. (1946). Audiovisual methods in teaching. Dryden Press. https://bit.ly/42aW03X
Link Google Scholar
Dalgarno, B., & Lee, M.J.W. (2010). What are the learning affordances of 3-D virtual environments? British Journal of Educational Technology, 41(1). https://doi.org/10.1111/j.1467-8535.2009.01038.x
Link DOI | Link Google Scholar
Dede, C. (2009). Immersive interfaces for engagement and learning. Science, 323(5910), 66-69. https://doi.org/10.1126/science.1167311
Link DOI | Link Google Scholar
Deng, Q., Wang, J., Hillebrand, K., Benjamin, C.R., & Soffker, D. (2020). Prediction performance of lane changing behaviors: A study of combining environmental and eye-tracking data in a driving simulator. IEEE Transactions on Intelligent Transportation Systems, 21(8), 3561-3570. https://doi.org/10.1109/TITS.2019.2937287
Link DOI | Link Google Scholar
Duchowski, A. T. (2002). A breadth-first survey of eye-tracking applications. Behavior Research Methods, Instruments, & Computers, 34(4), 455-470. https://doi.org/10.3758/BF03195475
Link DOI | Link Google Scholar
Farran, E., Formby, S., Daniyal, F., Holmes, T., & Herwegen, J. (2016). Route-learning strategies in typical and atypical development; eye-tracking reveals atypical landmark selection in Williams syndrome: Route-learning and eye-tracking. Journal of Intellectual Disability Research, 60(10), 933-944. https://doi.org/10.1111/jir.12331
Link DOI | Link Google Scholar
García-Carrasco, J., Hernández-Serrano, M.J., & Martín-García, A.V. (2015). Plasticity as a framing concept enabling transdisciplinary understanding and research in neuroscience and education. Learning, Media and Technology, 40(2), 152-167. https://doi.org/10.1080/17439884.2014.908907
Link DOI | Link Google Scholar
Gardony, A.L., Lindeman, R.W., & Brunyé, T.T. (2020). Eye-tracking for human-centered mixed reality: Promises and challenges. Proc.SPIE, 11310, 113100T. https://doi.org/10.1117/12.2542699
Link DOI | Link Google Scholar
Glennon, J.M., D’Souza, H., Mason, L., Karmiloff-Smith, A., & Thomas, M.S.C. (2020). Visuo-attentional correlates of Autism Spectrum Disorder (ASD) in children with Down syndrome: A comparative study with children with idiopathic ASD. Research in Developmental Disabilities, 104, 103678. https://doi.org/10.1016/j.ridd.2020.103678
Link DOI | Link Google Scholar
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. (2008). The WEKA data mining software: An update. SIGKDD Explor. Newsl., 11(1), 10-18. https://doi.org/10.1145/1656274.1656278
Link DOI | Link Google Scholar
Huang, H.M., Rauch, U., & Liaw, S.S. (2010). Investigating learners’ attitudes toward virtual reality learning environments: Based on a constructivist approach. Computers and Education, 55(3), 1171-1182. https://doi.org/10.1016/j.compedu.2010.05.014
Link DOI | Link Google Scholar
Lapborisuth, P., Koorathota, S., Wang, Q., & Sajda, P. (2021). Integrating neural and ocular attention reorienting signals in virtual reality. Journal of Neural Engineering, 18(6), 066052. https://doi.org/10.1088/1741-2552/ac4593
Link DOI | Link Google Scholar
Ma, X., Yao, Z., Wang, Y., Pei, W., & Chen, H. (2018). Combining brain-computer interface and eye-tracking for high-speed text entry in virtual reality. In Berkovsky & Y. Hijikata (Ed.), IUI ’18: 23rd International Conference on Intelligent User Interfaces (pp. 263-267). https://doi.org/10.1145/3172944.3172988
Link DOI | Link Google Scholar
Martinez, K., Menéndez-Menéndez, M.I., & Bustillo, A. (2021). Awareness, prevention, detection, and therapy applications for depression and anxiety in serious games for children and adolescents: Systematic review. JMIR Serious Games, 9(4), e30482. https://doi.org/10.2196/30482
Link DOI | Link Google Scholar
Mckinney, W. (2011). pandas: A foundational Python library for data analysis and statistics. Python High Performance Science Computer.
Link Google Scholar
Patney, A., Salvi, M., Kim, J., Kaplanyan, A., Wyman, C., Benty, N., Luebke, D., & Lefohn, A. (2016). Towards foveated rendering for gaze-tracked virtual reality. ACM Trans. Graph., 35(6), 1-12. https://doi.org/10.1145/2980179.2980246
Link DOI | Link Google Scholar
Pritchard, A. (2017). Ways of learning: Learning theories for the classroom. Routledge. https://doi.org/10.4324/9781315460611
Link DOI | Link Google Scholar
Rappa, N.A., Ledger, S., Teo, T., Wai Wong, K., Power, B., & Hilliard, B. (2022). The use of eye-tracking technology to explore learning and performance within virtual reality and mixed reality settings: A scoping review. Interactive Learning Environments, 30(7), 1338-1350. https://doi.org/10.1080/10494820.2019.1702560
Link DOI | Link Google Scholar
Rodero, E., & Larrea, O. (2022). Virtual reality with distractors to overcome public speaking anxiety in university students; [Realidad virtual con distractores para superar el miedo a hablar en público en universitarios]. Comunicar, 30(72). https://doi.org/10.3916/C72-2022-07
Link DOI | Link Google Scholar
Shadiev, R., & Li, D. (2022). A review study on eye-tracking technology usage in immersive virtual reality learning environments. Computers & Education, 104681. https://doi.org/10.1016/j.compedu.2022.104681
Link DOI | Link Google Scholar
Sun, Q., Patney, A., Wei, L.Y., Shapira, O., Lu, J., Asente, P., Zhu, S., McGuire, M., Luebke, D., & Kaufman, A. (2018). Towards virtual reality infinite walking: Dynamic saccadic redirection. ACM Transactions on Graphics, 37(4), 1-13. https://doi.org/10.1145/3197517.3201294
Link DOI | Link Google Scholar
Tanaka, Y., Kanari, K., & Sato, M. (2021). Interaction with virtual objects through eye-tracking. In M. Nakajima, J.G. Kim, W.N. Lie, & Q. Kemao (Eds.), International Workshop on Advanced Imaging Technology (IWAIT) 2021 (p. 1176624). SPIE. https://doi.org/10.1117/12.2590989
Link DOI | Link Google Scholar
Tavenard, R., Faouzi, J., Vandewiele, G., Divo, F., Androz, G., Holtz, C., Payne, M., Yurchak, R., Rußwurm, M., Kolar, K., & Woods, E. (2020). Tslearn, A Machine-learning Toolkit for Time Series Data. J. Mach. Learn. Res., 21, 118, 1-6.
Link Google Scholar
Wilson, M. (2002). Six views of embodied cognition. Psychonomic Bulletin and Review, 9, 625-636. https://doi.org/10.3758/BF03196322
Link DOI | Link Google Scholar
Wismer, P., Soares, S.A., Einarson, K.A., & Sommer, M.O.A. (2022). Laboratory performance prediction using virtual reality behaviometrics. PloS One, 17(12), e0279320. https://doi.org/10.1371/journal.pone.0279320
Link DOI | Link Google Scholar
Fundref
Technical information
Recebido: 26-12-2022
Revisado: 25-01-2023
Aceite: 23-02-2023
OnlineFirst: 30-05-2023
Data de publicação: 01-07-2023
Tempo de revisão do artigo: 30 dias | Tempo médio de revisão do número 76: -6 dias
Tempo de aceitação do artigo: 59 dias | Tempo médio de aceitação do número 76: 72 dias
Tempo de edição da pré-impressão: 142 dias | Tempo médio de edição pré-impressão do número 76: 155 dias
Tempo de processamento do artigo: 187 dias | Tempo médio de processamento do número 76: 200 dias
Métricas
Métricas deste artigo
Vistas: 56103
Leituras dos resumos: 54284
Descargas em PDF: 1819
Métricas completas do Comunicar 76
Vistas: 596153
Leituras dos resumos: 580422
Descargas em PDF: 15731
Citado por
Citas em Web of Science
Actualmente não há citações a este documento
Citas em Scopus
Actualmente não há citações a este documento
Ana Serrano-Mamolar, University of Burgos (Espanha)
