Measuring Artificial Intelligence Literacy in Higher Education: Validation of the ECAE-AI Scale

Abstract

The accelerated integration of artificial intelligence (AI) into higher education has increased the need for valid instruments capable of assessing AI literacy beyond technical skills, including attitudinal and ethical dimensions. Despite growing international interest in AI literacy, validated instruments adapted to Latin American undergraduate contexts remain scarce, limiting evidence-based educational interventions and institutional AI policies. This study aimed to validate the factorial structure of the ECAE-AI scale, a 14-item instrument designed to assess knowledge, attitudes, and ethical awareness regarding AI use among university students. A non-experimental, cross-sectional, instrumental design was applied to 408 undergraduate students from a northeastern Mexican university. Confirmatory factor analysis (CFA) using the DWLS estimator confirmed an excellent fit for the three-factor correlated model (CFI = .997; TLI = .996; RMSEA = .057; SRMR = .048). Standardised factor loadings ranged from .607 to .894. Reliability was excellent at both the global (? = .915; ? = .917) and dimensional levels. Convergent and discriminant validity were also supported. The findings support the ECAE-AI as a psychometrically robust instrument for assessing AI literacy in higher education, integrating cognitive, attitudinal, and ethical dimensions within a single framework. The scale provides practical value for diagnosing AI literacy profiles, designing evidence-based educational interventions, and supporting responsible AI integration policies in higher education institutions.