Competence learning in the European Higher Education area

Learning in HE must be based on the development of competencies. This was, and indeed continues to be, the challenge facing the European Higher Education Area (EHEA). As a result, every bachelor's and master's degree includes references to various kinds of skills (conceptual, procedural, and attitudinal), whose ultimate goal is to deliver quality academic training by offering students the best possible education. The final objective is to provide graduates with comprehensive development and employability, together with an interest in life-long learning (Šucha & Gamme, 2021). In this line, in 2020 the European Commission published a new framework of education to be applied over the period 2021-2025. Said communication attaches particular importance to achieving the aims mentioned previously. These goals have now been strengthened even further by the current pandemic (Commission to the European Parliament, 2020). The leitmotiv of the publication specifically refers to the need to train people to become independent and resilient so that they can make a high-level contribution to society. This goal is directly related to the changes that today’s society requires in the way teaching and learning are undertaken. Said need has been established as a common objective of European Union countries. Likewise, this cooperation has led to adaptation and improvement in education by strengthening the common proposals that centre on the need to boost digitalisation, both amongst teachers and students. In particular, this communication refers to the need to foster further use in the acquisition of digital strategies and skills in artificial intelligence [takeaway 10 and Action 11 (Sucha & Gamme, 2021)]. Likewise, special emphasis is placed on the development of cross-sectional skills related to critical thinking, entrepreneurship, creativity, and civic commitment. These skills are deemed fundamental if future generations of students, researchers and innovators are to build a proactive and resilient society. This report also highlights the need to address the issue of under-performance and student drop-out rates at university (Casanova et al., 2018). All of these intentions are directly related to the proposals for teaching innovation that focus on preventing the digital divide between teachers and students (Sáiz-Manzanares et al., 2021).

Given this situation, it is particularly important to address the need to train teaching staff in the acquisition of digital skills. In this line, the European Framework for the Digital Competence of Educators (DigCompEdu) distinguishes six levels of digital competence (Redecker & Punie, 2017): Newcomers (A1) are teachers who have had very little contact with the use of digital tools; Explorers (A2) are teachers who have taken their first steps in the use of digital tools, but who still lack a global approach; Integrators (B1) are teachers who use and experiment with digital tools for a range of purposes in an effort to discover which digital strategies work best in each context; Experts (B2) are teachers who use a series of digital tools with a certain degree of confidence, creativity and critical spirit in order to enhance their professional activities, and who are continually striving to broaden their repertoire of practices; Leaders (C1) are teachers who use a wide range of flexible, comprehensive and efficient digital strategies; and Pioneers (C2), are those teachers who question the appropriateness of current digital and pedagogical practices, in which they themselves are experts, whilst at the same time leading the way in innovation and standing as a benchmark for younger teachers.

These challenges, and the response of government decision-makers, lead to the conclusion that a major digital transformation is required in the teaching-learning process. Said transformation needs, on the one hand, a sound technological infrastructure (platforms, tools, interoperability) and, on the other, a pedagogical proposal (learning design, accreditation and assessment processes). This is why the structure of digital transformation would need to be addressed from a pyramid of actions. This would commence with education policy (strategies), identity and communication, and would then be followed by protocols of ethics, privacy and safety, academic adaptation of services, and would then continue with the design of the educational model. Finally, it would conclude by addressing curricular content (García-Peñalvo, 2021). This framework embraces a number of different models that can be used, prominent amongst which are: Technological Pedagogical Content Knowledge (TPCK) (Koehler & Mishra, 2009), a model which points towards interaction that is at the same level of importance amongst the pedagogical elements, the use of technological resources and a proposal of educational content. Another key model is the Substitution, Augmentation, Modification and Redefinition model (SAMR) (Garcia-Utrera et al., 2014), which prioritises the importance of technology in educational change. In this study, we apply the TPCK (Koehler & Mishra, 2009), since we believe that the weight of technology and of the pedagogical model used during the instructional process should have the same degree of relevance to achieve successful and effective learning amongst students.

Learning in virtual environments in higher education

Teaching in HE, whether face-to-face or blended, is increasingly taking place in virtual learning environments or in a Learning Management System (LMS). This has grown due to the current health crisis sparked by the SARS CoV-2 virus (García-Peñalvo, 2021). The pandemic has accelerated the digitalisation of teaching, since a large part of the teaching is currently delivered either completely online (e-learning), or partially online (blended learning or b-learning), depending on the outbreaks or spikes of the virus. This has major consequences for teaching staff vis-à-vis designing and implementing their teaching, which in turn affects student satisfaction with the process (Tang et al., 2021).

Recent studies (Leal-Filho et al., 2021) have shown that the pandemic caused by COVID-19 has brought about a global change in both teaching and learning. Perceived user (teachers and students) satisfaction is seen to vary depending on the confinement conditions imposed and the technological resources available (kind of device and connection networks). As a result, these authors highlight the need to implement policies that foster teaching innovation and improvements in technological resources if the challenge is to be met. Other studies carried out with health science students (medicine and nursing) have shown that satisfaction with online teaching during the early stages of the pandemic was higher amongst teachers than amongst students (Li et al., 2021). Moreover, in this latter group, differences were reported in the level of satisfaction depending on whether the situation affected practical clinical training.

Differences were also found depending on the country in question. For example, students in China and India reported a high level of satisfaction (80.29%) compared to a lower level amongst students in Jordan (26.77%). The authors also found that the level of student satisfaction was affected by their personal situation, their lockdown situation, socio-emotional support (family or friends), technological resources (network, laptops, etc.). Finally, teacher satisfaction was also influenced by the perceived satisfaction amongst their students. Another key study (Tang et al., 2021) into teaching during the pandemic underscored the importance of paying attention to detail when approaching pedagogical-technological issues in teaching contexts, stressing the need to include more virtual activities in order to boost student motivation and foster interaction with them.

Within this framework, one important aspect vis-à-vis achieving satisfaction with the teaching-learning process of all the stakeholders involved concerns the design of LMSs (García-Peñalvo, 2021). Recent studies (Chakma et al., 2021) have reported that LMSs which include a metacognitive design foster student development of self-regulated learning (SRL) that is creative and autonomous. The use of videos is also emerging as an extremely valuable resource to promote effective learning during the current pandemic (Kidess et al., 2021). Experts have also posited the need to continue using these resources even when the health situation eventually improves (Lowe et al., 2021). Another highly effective tool is the use of tele-simulation, particularly in health science degrees (Diaz & Walsh, 2021). This strategy differs in certain aspects from experiences with virtual reality and virtual laboratories (Sáiz-Manzanares et al., 2021). In these experiences, various kinds of learning strategies related to developing students’ cognitive, metacognitive, behavioural, motivational and technical skills are put into practice.

The use of flipped learning experiences applied to online project based learning (OPBL) (Sarwa et al., 2021) is also proving to be highly effective. Implementing this kind of experience has been linked to a high degree of student satisfaction with the teaching-learning process (Sáiz-Manzanares et al., 2021). All of these resources share a common feature; the design of a teaching environment that provides the student with deep learning opportunities. In addition to incorporating all of these innovative methodological tools, teachers need to have means available that include data visualisation techniques which allow them to monitor the teaching-learning process in virtual environments (Sáiz-Manzanares et al., 2021).

These tools should embrace a high degree of usability and include data analysis systems, within what has been called Educational Data Mining (EDM) and Artificial Intelligence (Bonami, 2020). Use of these resources will enable teachers to familiarise themselves with students’ learning patterns and, based on these, to create individually tailored itineraries. Nevertheless, mixed research methods will be required in order to carry out such analyses (in other words, those which value both quantitative and qualitative aspects) (Anguera et al., 2018). This use will enable a better educational response to be provided to each student’s learning needs (García-Perales & Almeida, 2019; Salinas-Ibáñez & De-Benito, 2020).

In sum, resources for virtualisation, measurement and analysis will be included in what has been termed Advanced Learning Technologies (ALT). In turn, ALT will enable the development of Self-Regulated Learning (SRL) and enhance student motivation (Azevedo & Gasevic, 2019). If all of these resources are to be applied correctly, however, training in digital skills by all the stakeholders involved in the teaching-learning process will be required (Redecker & Punie, 2017; Unesco, 2019). As an example of this a recent study (Sáiz-Manzanares et al., 2021) indicated that teaching staff with a B2 or C1 level in digital skills only put into practice 66.6% of the available resources offered in an LMS such as a Modular Object-Oriented Dynamic Learning Environment (Moodle). This is a key aspect to be considered by heads of university institutions, since it points the way towards a digital transformation aimed at promoting the development of training programmes for teachers and students in digital skills (García-Peñalvo, 2021).

In addition, teaching based on student experimentation and active practice needs to be fostered even further. This ties in directly with objective 4 (Quality Education) of the Agenda 2030 (United Nations. Department of Economic and Social Affairs, 2021). At the same time, the methodological and technological change in HE teaching, which was already a challenge for society in the 21^st century, has been hastened due to the current pandemic. As a result, teachers should include in their teaching design various e-learning or b-learning teaching resources that allow them to successfully face up to this challenge.

Participants

We worked with a sample of 225 third year bachelor degree students who were taking a degree in health sciences (the Degree in Occupational Therapy, and the Degree in Nursing). The courses in which this study was carried out were taught in the second semester and lasted nine weeks. In order to control the type of teacher variable, the same teacher taught both courses. The teacher was an accredited expert in virtual teaching from the University of Burgos and her level of online teaching could be classified as a C1 skill level (Leaders (C1) in accordance with the DigCompEdu (Redecker & Punie, 2017). Convenience sampling was used to select the sample. Two groups were established: Group 1, in which teaching was given during the first year of the pandemic (academic year 2019-2020) using an e-learning teaching method, and Group 2, in which teaching was given in the second year of the pandemic (academic year 2020-2021) applying a b-learning teaching method. Descriptive statistics for the mean (M) and standard deviation (SD) of age broken down by gender can be seen in Table 1. This table shows there is a greater percentage of females than males, which is common in health science degrees. This is reflected in the latest university figures report issued by the Conference of Rectors of Spanish Universities (CRUE), which shows that the mean percentage of females in health science degrees is 73.8% (Hernandez-Armenteros & Pérez-García, 2018).

Tools

a) UBUVirtual platform. This platform is an LMS, developed in Moodle. Version 3.9 was used.

b) Scale of learning strategies (ACRA for its initials in Spanish) (Román-Sánchez & Gallego-Rico, 2008). This scale has been widely applied in research focusing on learning strategies in Spanish-speaking environments (Carbonero et al., 2013). The ACRA scale identifies 32 strategies at different phases of information processing. In this study, the scale of metacognitive strategies was used, which includes the subscales of self-knowledge, self-planning and self-assessment. ACRA has a total Cronbach Alpha reliability coefficient equal to α=0.90 and for the scale of metacognitive strategies of α=0.89, a construct validity between evaluators of r=0.88, and a content validity of r=0.88. For this study, for each subscale, we found global reliability indicators for the scale of metacognitive strategies α=0.88, and for the subscales of self-knowledge α=0.82, self-planning α=0.86, and self-assessment α=0.80, respectively.

c) Design of the course. We applied a teaching method grounded on the use of online project based learning (OPBL) in LMS UBUVirtual. This involved applying the ABP method in e-learning environments. In other words, students draw up a project together in LMS. The materials used to apply this method can be seen in Sáiz-Manzanares (2018).

d) Two virtual laboratories that included multimedia resources were used. These laboratories are open access and can be examined in the Repository of the University of Burgos (Degree in Occupational Therapy. Course: Early Stimulation: https://bit.ly/36U2ysn and https://bit.ly/3rykbr9; Degree in Nursing. Course: Quality Management: https://bit.ly/3rtuXyU).

e) We used flipped learning experiences that are open access and which may be consulted in the Repository of the University of Burgos (Degree in Occupational Therapy https://bit.ly/2TDSiBF and https://bit.ly/2UMAxAK. Degree in Nursing https://bit.ly/3rs01is and https://bit.ly/3rtXTGW).

f) Satisfaction survey with the teaching-learning process (ESPEA) (Sáiz-Manzanares, 2018). This is a survey drawn up ad hoc and which comprises 20 closed item responses measured on a Likert type scale from 1 to 5 points. The reliability indicators for the whole scale were α=0.93, and for each element in the scale these were between α=0.92 and α=0.94. Also included, are four open questions that refer to aspects to be: changed, extended, reduced, and improved.

Procedure

Prior to conducting the study, it was approved by the University of Burgos Bioethics Committee (No. IR 30/2019). During the first week of the academic year, students signed the informed participation consent form. In Group 1, teaching was delivered during the first year of the COVID-19 pandemic online after the fourth week of the semester, following the government’s declaration of the state of emergency, which implied using an e-learning teaching method. In Group 2, teaching was given during the second year of the pandemic. Teaching was provided through a b-learning method (lessons were given virtually through the Teams platform and practical lessons were given on site, adopting safety measures such as the use of FFP2 and FFP3 face masks and protective screens). All the courses involved the use of the OPBL method, virtual laboratories and flipped classroom experiences (see Tools section).

Research designs

A quantitative study was carried out in which we applied a 2x2 factorial design (type of academic year and type of degree) (Campbell & Stanley, 2005). A qualitative study was also carried out in which a comparative design was applied (Flick, 2014).

Data analysis

Quantitative study: prior analyses were carried out to verify the normality of the sample (asymmetry and kurtosis statistics) and group homogeneity, prior to intervention [descriptive statistics (mean and standard deviation), one-factor fixed effects ANOVA (first year vs. second year of the pandemic)]. The hypotheses were then tested, for which a one-factor fixed effects ANCOVA was used (first year in which teaching was given (e-learning teaching method) vs. second year (b-learning teaching method) and co-variate type of degree (Degree in Occupational Therapy vs. Degree in Nursing). All of the calculations were carried out using the SPSS v.24 statistical package (IBM Corp, 2016).

Qualitative study: the answers to the open ESPEA questions (Sáiz-Manzanares, 2018) were categorised. Co-occurrence analysis between the categorised answers and the open response documents was then applied. ATLAS.ti 9 software (Atlas.ti, 2020) was used for analysis.

Prior analysis

Prior to testing the hypotheses, we checked to see whether the sample distribution complied with the parameters of normality, for which we found the indicators of asymmetry and kurtosis compared to the results in the ACRA Scale of Metacognitive Strategies (Román-Sánchez & Gallego-Rico, 2008), a test which was applied to the various groups prior to commencing intervention. No extreme values were found for asymmetry (extreme values are considered to be those above |2.00|) or for kurtosis (extreme values are those between 8.00 and 20.00) (Bandalos & Finney, 2001), so that the sample can be said to display a normal distribution (Table 2). As a result, parametric contrast tests were applied in the quantitative study.

We then carried out a one-factor fixed effects ANOVA to test whether, prior to intervention, there were any significant differences between the groups in the results of the Metacognitive Strategies Scale (Román- Sánchez & Gallego-Rico, 2008). Differences between means in the two groups of students were minimal (from 0.1 to 0.3), and no statistically significant differences were found (Table 3), so that the groups were considered to be homogeneous prior to intervention.

Quantitative analysis

In order to test the RQ1, a one-factor fixed effects ANCOVA was performed [teaching given in the first year of the pandemic (e-learning teaching) vs. second year of the pandemic (b-learning teaching) and covariate (type of degree)]. Significant differences were found in ESPEA (Sáiz-Manzanares, 2018) with regard to the independent variable first academic year (e-learning teaching) vs. second year of the pandemic (b-learning teaching) in items 10 (expectations of the course), 18 (comparison with other courses), 19 (evaluation of the virtual laboratories), and 20 (general satisfaction), in all cases in favour of the group with e-learning teaching. An effect of the covariate value type of degree was also found in items 8 (addressing all elements of the teaching guide), 9 (help with job placement), 17 (satisfaction with the dynamics of the course), 18 (comparison with other courses) and 19 (evaluation of virtual laboratories). This effect was, however, seen to be low in all cases (see Table 4 in https://bit.ly/2UFS5OD). In other words, differences were minimal in the means of the two groups.

In order to test the RQ2, a one-factor fixed effects ANCOVA was performed (first year that teaching was given (e-learning teaching) vs. second year (b-learning teaching) and covariate (type of degree)). Results point to significant differences in the independent variable, academic performance, in favour of Group 2 (b-learning teaching) and no effects of the covariate type of degree were found (Table 4). In this case, the differences between the means were also minimal and the size of the effect was low.

Qualitative analysis

With regard to the analysis of the responses to the open questions in ESPEA (Sáiz-Manzanares, 2018), 16 documents were analysed which included answers concerning aspects to be changed, extended, reduced and/or improved. Students’ answers were subsequently categorised, distinguishing between the type of degree and the year of the pandemic in which teaching was given. Answers were grouped into two analysis criteria (i) aspects to be improved, and (ii) most highly valued aspects. ATLAS.ti 9 was used to process and analyse data. A co-occurrence analysis was then performed between the categorisations by academic year and the documents. Areas that could be improved in the Occupational Therapy Degree related to carrying out real practical work with children, an aspect which was highlighted both in the first as well as in the second year of the pandemic, and facilitating decision making vis-à-vis therapeutic intervention, an aspect which was only pointed out in the first year of the pandemic (e-learning teaching).

In the Nursing Degree, no suggestions were put forward for the first year of the pandemic (e-learning teaching) but were for the second (b-learning teaching). These suggestions concerned doing workshops during the practical training in order to explain the teaching-learning methodology and reducing the syllabus a little. Regarding the aspects considered to be most positive by students, 80% felt that the courses were well-structured, and that the syllabus was comprehensive and fitted well with the time devoted to teaching. The method used was deemed to be innovative and was seen by students as very positive in terms of their learning. Nevertheless, they felt that this type of method demanded more work time from them and that it included many technological resources they were not accustomed to using.