Volume index - Journal index - Article index - Map ---- Back
Higher education, learning, motivation, personal learning environments, cooperative learning environments, virtual learning, educational innovation, educational research
MOOCs have featured prominently in the scientific literature recently as a new way to provide training which is attracting millions of students across the world and forcing universities to reformulate their online education courses. MOOCs are seen to represent the next evolution in e-learning within a continuum which, according to Conole (2014), spans the earliest movements in multimedia in the 1980s to the MOOC that first emerged in 2008 followed by Learning Analytics two years later. The scale of MOOCs, the speed at which they have grown and the difficult questions they pose are increasingly prominent as the purpose of Higher Education and the very future of the university comes under scrutiny. It clearly indicates that something new is happening, something more than a mere trend. So, this is a subject of particular concern to anybody seriously interested in the digital future of education.
The interest this subject arouses is seen in a body of research centred on various MOOC projects. Works by Liyanagunawardena, Adams & Williams (2013), Castaño (2013) and Karsenti (2013) were systematic studies of investigations into MOOCs between 2008 and 2013. The main lines of research included the pedagogical design of MOOCs, interaction between students and the perspectives for learning and its associated variables (motivation, attitudes and perspectives). Other aspects were cost, universal accessibility to Higher Education and the problem of student dropout rates.
Many authors state that MOOC are substandard in terms of educational rigour (Vardi, 2012; Zapata-Ros, 2013), and that the current discourse on MOOCs merely reflects strategic, institutional, economic, social and technological concerns whereas there is no real discussion of the courses’ pedagogical value (Guàrdia, Maina & Sangrà, 2013: 4).
Despite these criticisms, and the fact that MOOC constitute a type of education that is flexible but not widely standardized (Shirky, 2013), various authors suggest there is a difference between c-MOOC (connectivist) and the more traditional x-MOOC (Downes, 2011; Siemens, 2012a; Rodriguez, 2013), making it impossible to talk of a single pedagogical design. Rodriguez (2012) analysed several courses of both tendencies and established that the difference between the two lies in their theory of learning and pedagogical model.
This initial description was too simplistic and is now more complex. Knox, Bayne, Macleod, Ross and Sinclair (2012) attempted to overcome these deficiencies by incorporating more interesting and innovative e-learning practices into their «E-learning and Digital Cultures» course, giving preference to content submission to the social networks of the process, the community and learning.
This proposal is in line with Lane (2012) who encountered difficulties in situating her approach within the «Stanford Model» versus the «Connectivist MOOC» debate, and proposed her own task-based project called s-MOOC (skills-MOOC).
This simplistic x-MOOC and c-MOOC classification has been bypassed thanks to alternative descriptions of the nature of MOOCs. Downes (2013) suggests four criteria: autonomy, diversity, openness and interactivity. Going further, Clark (2013) deploys a taxonomy of eight different MOOC types, claiming that they can be located at any point along the spectrum of traditional online courses. Conole (2013) proposes that they be classified as a set of 12 dimensions, which makes MOOC design even more complex.
In this sense cooperative MOOCs try to respond to MOOC student heterogeneity by producing an X-type course that nevertheless incorporates the advantages of connectivist courses (Fidalgo, Sein-Echaluze & García Peñalvo, 2013): intensive use of social networks, creation of learning communities (Alario-Hoyos & al., 2013) and the deployment of PLE, or personal learning environments (Castaño & Cabero, 2013: 102).
The efficacy of online training and MOOCs continue to represent an evolution in e-learning (Conole, 2014), and this theme is well-established in numerous research meta-analyses (Cabero, 2008; Means, Toyama, Murphy, Bakia & Jones, 2010). Although several studies have indicated that the pedagogical foundations of MOOCs are solid according to the various formats they employ (Glance, Forsey & Riley, 2013; Sonwalkar, 2013), the influence of different MOOC designs on learning outcomes has yet to be adequately researched. References to this theme are found in connectivist courses, the only in-depth studies are by Kop & Fournier (2011), Kop, Fournier & Mak (2011) and De Waard (2011, 2013), focussing more on their transformative effects on the conventional structures of knowledge generation than rigorously measuring learning outcomes.
From another viewpoint, some studies point to the potential of MOOCs to spur student autonomy (Kop, Fournier & Mak, 2011; Milligan, Littlejohn & Margaryan, 2014) and develop 21st century competences (Yeager, Hurley-Dasgupta & Bliss 2013; Sangrá & Wheeler, 2013).
Motivation has already been identified by Milligan, Littlejohn & Margaryan (2013) as a variable that enhances participation and academic success among students. Similar studies have recently appeared, such as Cheng (2014) on emotional competence in MOOC students, and Veletsianos (2013) on student learning experiences on MOOCs.
Research has also given rise to more sceptical voices on the use of MOOCs in Higher Education. Some authors show that the advantages of MOOCs are no different from those that were already known from distance learning (Fini, 2009; Yuan & Powell, 2013; Harder, 2013).
It is also known that student dropout rates have increased with the appearance of MOOCs. However, as Liyanaguna wardena, Adams & Williams (2013) point out, data on MOOC dropout rates are not readily available. Jordan (2013) examined 24 MOOCs and found that the highest rate of course completion was 19.2% while most barely reached 10%. Liyanagunawardena, Parslow & Williams (2014) show that MOOC students do not typically drop out for financial reasons since they do not pay enrolment or tuition fees; these authors suggest that abandonment has more to do with dissatisfaction at not achieving personal objectives.
The aim of this investigation is to analyse the pedagogical design of a cooperative MOOC and measure its influence on student motivation and academic results. The questions addressed are:
a) Is there a relationship between academic performance and the pedagogical design of the course?
b) Is there a relationship between student motivation and the pedagogical design of the course?
c) Is there a relationship between academic performance and student motivation?
One of the mainstays of this research was the pedagogical design of the course which was cooperative in nature. For the design of the MOOC, we carried out a Delphi double to string study with 53 experts in e-learning and ICTs from European and Latin American universities. We asked them about MOOC types, ways of learning and assessment methods, and the roles and functions of tutors. We took their responses and resubmitted the design of the course, located on the Chamilo Metauniversity open source e-learning platform under GNU/GPLv3 licensing.
In addition, and to foment interaction between participants, we actively encouraged students to use social networks (Twitter, Skype, blogs, Facebook, LinkedIn, etc.). The platform’s technical limitations were overcome by opening a Ning website to allow students to socially interact and discuss their contributions. They did e-activities on a weekly basis, all of which helped participants to construct their own PLE.
The statistical analysis of the data was carried out with the SPSS version 22 program. In the data collection process, besides the assessment of the e-activities by the tutors, there was a set of four questions on the course design with responses measured on the Likert scale. The data on motivation was gathered by means of an IMMS (Instructional Materials Motivation Survey) which the MOOC students completed at the end of the course. This was a Likert-type questionnaire made up of 36 items divided into four categories (attention, confidence, satisfaction and relevance) based on Keller’s ARCS motivation model (1987). In this case we took the proposal of Di Serio, Ibáñez & Delgado (2013) with a documented reliability coefficient of 0.96 and adapted it slightly to fit MOOCs.
The course was designed for students in the fourth year of a Primary Education degree course at the Universidad del País Vasco, although the very nature of the MOOC meant that it was accessible on the Net to all those interested in this subject, in line with other MOOC experiences (Siemens, 2012b; Knox, Bayne, Macleod, Ross & Sinclair, 2012).
Of the 744 students who enrolled on the MOOC the sample consisted of 186 participants, classified as N in the research. In terms of the dropout rate, 186 students began the course, 25.83% of those who had signed up, and 88 completed the course, 11.82%. Sample attrition was in line with general MOOC dropout rates although participation was slightly higher than the 10% indicated by Jordan (2013).
We present the analysis of the data yielded by the scales used (IMMS and the scale for the course design) in the order of the research questions posed. The global results of the correlational analysis also come with a study based on the division of the MOOC students into two groups according to age, those who are 31 or under and those over 31, since the first group was formed of undergraduates and the second group was not.
The course design is based on four variables assessed by the participants using the Likert scale applied to these four items: the use of small video packages is a good idea; I was able to control the development of the course with ease thanks to carrying out e-activities; interaction with course colleagues via the network enhances learning; the use of a social network as a course complement has helped me to follow the course.
In reference to the first research question (Is there a relationship between academic performance and the pedagogical design of the course?), a direct relationship between these two factors is observed. The relationship is significant both in the overall result and in the results for the two age groups (table 1). The four items on the scale that relate to the course design were valued positively or very positively by 85% of the students.
The second research question (Is there a relationship between student motivation and the pedagogical design of the course?) emphasises the potential relationship between motivation and course design. The results in table 2 show a direct link between the type of course design and student motivation. The correlational analysis of the total sample is significant in this global aspect and is supported by the significance that emerges for each of the factors (attention, confidence, satisfaction and relevance) on which the level of motivation is based, according to the IMMS scale. These results are significant and they appear in all the factors and among all the students in the two age groups, just as occurred with the first research question. Thus, we can state that the course design influences student motivation.
Thus we can state that there exists a direct relationship between each of the four factors that measure motivation and course design.
With regard to the attention variable, the results (table 3) confirm that the items related to methodology (items 2 and 8), quality (11, 12, 15, 28 and 29), organization (17, 20 and 31) and use of material (22 and 24), as well as interaction via Internet (19), all positively contribute to student motivation on the MOOC.
In addition, there is an increase in each individual’s confidence in learning, as gathered in the second factor on the IMMS scale (items 1, 3, 4, 7, 13, 25, 34 and 35 in table 3)
The same result is found in the factors that correspond to satisfaction and relevance (table 2). According to the data (table 4), the course design (items 6, 27, and 36) together with the material (9, 16, 18, 23 and 33), and the development of the MOOC (5, 10, 21 and 32) is relevant and satisfactory (14, 26 and 30).
The results for the third research question (Is there a relationship between academic performance and student motivation?) were different in terms of global perspective and age group.
The following results show that although there is no direct overall relationship between academic performance and motivation, there is some significance in the global relationship between performance and one of the IMMS factors, this being student satisfaction (table 5).
This significance is also seen in the students aged over 31 yet it has no correlation in the younger age group, 31 and under (table 5).
These results lead us to state that there is no direct relationship between global motivation and performance, yet there exists such a relationship between global motivation and one of the factors that constitutes motivation, namely satisfaction. Overall, satisfaction correlates directly to student performance on the MOOC.
Finally, an analysis was made of the mediation of the satisfaction factor in the relationship between course design and performance in the total sample. With the introduction of the mediatory variable (satisfaction) the relationship between the design factor and performance disappears and ceases to be significant, which means that the relationship between design and performance is based on the satisfaction factor because, when it is controlled, the previous relationship is nullified: b = 0.25* (E.T. = 0.10) / b = 0.16ns (E.T = 0.12) (*p < .01). Hence, satisfaction mediates between design and performance, so the greater the satisfaction with the design of the course, the better the student performance.
MOOCs are defined by the huge number of students they attract and their heterogeneity, and also by high dropout rates. The study shows that a mixed course design that is cooperative in nature and which incorporates social networks as a learning strategy can help to reduce this phenomenon (Fidalgo, Sein-Echaluze & García Peñalvo, 2013). These data reinforce the validity of interaction for learning on online courses, as shown in previous research (Vidal & Camarena, 2014).
Course design influences performance since there is a direct significance between both factors in the global result and in the characteristics of the pedagogical design proposed (use of micro-content, video micro-packages, intensive social network activity and the carrying-out of online activities within students’ own PLE).
Likewise there is a direct link between course design and the four motivational factors on the IMMS scale: attention, confidence, satisfaction and relevance. This connection affects methodology and the quality and organization of the written and multimedia material used on the course as well as the interaction between students. These data reinforce the potential of cooperative designs for learning in Higher Education.
Yet this does not occur when the scale and its four motivational factors correlate to academic performance. There is no global significance between motivation and performance, but we observe that one of the factors on the IMMS scale, satisfaction, indicates that there is a direct link to performance in the global results and in references to students over 31 years old. At a time when universities are rethinking their online training courses, traditional MOOCs may attract new students to university but it is cooperative MOOCs that can increase the level of student satisfaction and help cut dropout rates. Possibly a hybrid design like the one proposed here fits better with the different types of student that take MOOCs (Milligan, Littlejohn & Margaryan, 2013), and enables them to reach their personal objectives (Liyanagunawardena, Parslow & Williams, 2014). This would allow universities to offer continuous training and Lifelong Learning with course designs that better adapt to students’ needs. Indirectly, it also reveals the capacity of this pedagogical design to develop in students those learning competences that are essential in the 21st century.
Finally, the general sample’s level of satisfaction derived from the positive perception of a good course design and the consequent, good academic performance achieved by those students supports the use of massive open online courses in graduate studies.
After the analysis of the mediation of the satisfaction factor in the relationship between the pedagogical design of the course and performance, we observe that the relationship between both ceases to be significant. It is, therefore, the level of satisfaction regarding the pedagogical design of the course that influences academic outcomes. It would be interesting to broaden these results with an analysis of students’ learning experiences on MOOCs.
This research project was financed by the UPV/EHU: «Rendimiento y nivel de satisfacción de los participantes en un curso on-line masivo y abierto (MOOCs)» (Performance and level of student satisfaction with massive open online courses), reference number: EHU 13/59, granted in 2013.
Alario-Hoyos, C., Pérez-Sanagustín, M., Delgado-Kloos, C., Parada, H. A., Muñoz-Organero, M. & Rodríguez-de-las-Heras, A. (2013). Analysing the Impact of Built-in and External Social Tools in a MOOC on Educational Technologies. In D. Hernández-Leo, T. Ley, R. Klamma & A. Harrer (Eds.), Scaling Up Learning for Sustained Impact (pp. 5-18). Berlin, Germany: Springer Berlin Heidelberg. (DOI: http://doi.org/tkh).
Cabero, J. (Dir.) (2008). E-learning: Metaanálisis de investigaciones y resultados alcanzados. Informe final. (http://goo.gl/0ayp3E) (30-04-2014).
Castaño, C. (2013). Tendencias en la investigación en MOOC. Primeros resultados. (http://goo.gl/mBKuTi) (10-12-2013).
Castaño, C. y Cabero, J. (Coords.) (2013). Enseñar y aprender en entornos m-learning. Madrid: Síntesis.
Cheng, J.C.Y. (2014). An Exploratory Study of Emotional Affordance of a Massive Open Online Course. European Journal of Open, Distance and e-Learning 17, 1 43-55. (http://goo.gl/sJuxAh) (15-03-2014).
Clark, D. (2013). MOOC: Taxonomy of 8 Tipes of MOOC. (http://goo.gl/VYA9XH) (09-12-2013).
Conole, G. (2013). Los MOOC como tecnologías disruptivas: estrategias para mejorar la experiencia de aprendizaje y la calidad de los MOOC. Campus Virtuales, 2, 16-28. (http://goo.gl/EK9ZPl) (10-04-2014).
Conole, G. (2014). Reviewing the Trajectories of E-learning. (http://goo.gl/Ferxef) (08-01-2014).
De Waard, I. (2013). Analyzing the Impact of Mobile Acces on Learner Interactions in a MOOC. A Thesis Submitted to the Faculty of Graduate Studies. Centre for Distance Education. Athabasca University. (http://goo.gl/vWYLoe) (28-04-2014).
De Waard, I., Abajian, S. & al. (2011). Using mLearning and MOOC to Understand Chaos, Emergence, and Complexity in Education. International Review of Research in Open and Distance Learning, 12 (7) (http://goo.gl/wxdMZt) (28-04-2014).
Di Serio, A., Ibáñez, B. & Delgado, C. (2013). Impact of an Augmented Reality System on Students´Motivation for a Visual Art Course. Computers & Education, 68, 586-596. (http://goo.gl/s3zBcp) (DOI: http://doi.org/tkq) (09-04-2014).
Downes, S. (2011). ‘Connectivism’ and Connective Knowledge. (http://goo.gl/q1eEht) (25-04-2014).
Downes, S. (2013). Week2: The Quality of Massive Open Online Courses. (http://goo.gl/W57f7A) (14-04-2014).
Fidalgo, A., Sein-Echaluce, M.L. & García-Peñalvo, F.J. (2013). MOOC cooperativo. Una integración entre cMOOC y xMOOC. In A. Fidalgo, M.L. Sein-Echaluce (Eds.), Actas del II Congreso Internacional sobre Aprendizaje, Innovación y Competitividad, CINAIC. (pp. 481-486). Madrid, España: Fundación General de la Universidad Politécnica de Madrid. (http://goo.gl/oxA06L) (09-04-2014).
Fini, A. (2009). The Technological Dimension of a Massive Open Online Course: The Case of the CCK08 Course Tools. International Review of Research in Open and Distance Learning, 10 (5). (http://goo.gl/xuBAOV) (29-04-2014).
Glance, D.G., Forsey, M. & Riley, M. (2013). The Pedagogical Foundations of Massive Open Online Courses. First Monday, 18, 5. (http://goo.gl/WVQl1C) (DOI: http://doi.org/tkp).
Guàrdia, L., Maina, M. & Sangrà, A. (2013). MOOC Design Principles: A Pedagogical Approach from the Learner’s Perspective. (http://goo.gl/G4Rjxt) (15-12-2013).
Harder, B. (2013). Are MOOC the Future of Medical Education? BMJ, 346. (DOI: http://doi.org/tks).
Jordan, K. (2013). MOOC Completion Rates: The Data. (http://goo.gl/73AxVf) (15-12-2013).
Karsenti, T. (2013). The MOOC. What the research says. International Journal of Technologies in Higher Education, 10 (2), 23-37.
Keller, J. M. (1987). Strategies for Stimulating the Motivation to Learn. Performance and Instruction. 26 (8), 1-7.
Knox, J., Bayne, S., Macleod, H., Ross, J. & Sinclair, C. (2012). MOOC Pedagogy: the Challanges of Developing for Coursera. (http://goo.gl/z8pQbP) (03-12-2013).
Kop, R. & Fournier, H. (2011). New Dimensions to Self-directed Learning in an Open Networked Learning Environment. International Journal of Self-Directed Learning, 7, 2, 1-18.
Kop, R., Fournier, H. & Mak, S.F.J. (2011). A Pedagogy of Abundance or a Pedagogy to Support Human Beings? Participant Support on Massive Open Online Courses. International Review of Research in Open and Distance Learning, Special Issue - Emergent Learning, Connections, Design for Learning, 12, 7, 74-93.
Lane, L. (2012). Three Kinds of MOOC. (http://goo.gl/lwTkTA) (10-12-2013).
Liyanagunawardena, T.R., Adams, A.A. & Williams, S.A. (2013). MOOC: A Systematic Study of the Published Literature 2008-12. The International Review of Research in Open and Distance Education, 14, 3, 202-227. (http://goo.gl/GMqIB2) (01-03-2014).
Liyanagunawardena, T.R., Parslow, P. & Williams, S.A. (2014). Dropout: MOOC Participants’ Perspective. In U. Crees & C. Delgado (Proceedings Editors), Proceedings of the European MOOC Stakeholder Summit 2014, 95-100 (http://goo.gl/8BEVHM) (25-04-2014).
Means, B.,Toyama, Y., Murphy, R., Bakia, M. & Jones, K. (2010). Evaluation of Evi-dence–based Practices in Online Learning: A meta-analysis and Review of Online Learning Studies. Washington, D.C.: U.S. Department of Education. (http://goo.gl/Yh6tgU) (27-04-2014).
Milligan, C., Littlejohn, A. & Margaryan, A. (2014). Workplace Learning in Informal Networks. Journal of Interactive Media Environments, 0. (http://goo.gl/tqfHaq) (30-04-2014).
Milligan, C.; Littlejohn, A. & Margaryan, A. (2013). Patterns of Engagement in Connectivist MOOC. Journal of Online Learning and Teaching, 9(2), 149-159. (http://goo.gl/7ALBbo) (30-04-2014).
Rodriguez, C.O. (2012). MOOC and de AI-Stanford like courses: Two Successful and Distinct Course Formats for Massive Open Online Courses. European Journal of Open, Distance and E-Learning, 2012 (II). (http://goo.gl/Jt8Wfv) (25-04-2014).
Rodriguez, C.O. (2013). The Concept of Openness Behind c and x-MOOC (Massive Open Online Courses). Open Praxis, 5 (1), 67-73. (http://goo.gl/JwG34l) (14-04-2014).
Sangrà, A. & Wheeler, S. (2013). New Informal Ways of Learning: Or Are we For-malising the Informal? Revista de Universidad y Sociedad del Conocimiento, 10 (1), 286-293. (http://goo.gl/Z7mSi7) (DOI: http://doi.org/tkt) (30-04-2014).
Shirky, C. (2013). MOOC and Economic Reality. (http://goo.gl/gw3fWZ) (28-04-2014).
Siemens, G. (2012a). MOOC are Really a Platform. (http://goo.gl/Jt8Wfv) (18-04-2014).
Siemens, G. (2012b). What is the Theory that Underpins ‘our’ MOOC? (http://goo.gl/-G7IJh6) (28-03-2014).
Sonwalkar, N. (2013). Why the MOOC Forum now? MOOC Forum, 1 (1). (http://goo.gl/LXbBhq) (30-04-2014).
Vardi, M.Y. (2012). Will MOOC Destroy Academia? Communications of the ACM, 55 (11), 5. (http://goo.gl/hpnvWe) (DOI: http://doi.org/tkv) (30-04-2014).
Veletsianos, G. (2013). Learner Experiences with MOOC and Open Online Learning. Hybrid Pedagogy. (http://goo.gl/frOkJy) (15-04-2014).
Vidal, A.A. & Camarena, B.O. (2014). Retos y posibilidades de los cursos en línea a partir de una experiencia concreta. Pixel-Bit, 44, 19-34. (http://goo.gl/UXXDBy) (DOI: http://doi.org/tkw) (15-04-2014).
Yeager, C., Hurley-Dasgupta, B. & Bliss, C.A. (2013). cMOOC and Global Learning: An Authentic Alternative. Journal of Asynchronous Learning Networks, 17 (2), 133-147. (http://goo.gl/uu3FWH) (30-04-2014).
Yuan, L. & Powell, S. (2013). MOOC and Open Education: Implications for Higher Education. Cetis White Paper 2013: WP03. (http://goo.gl/i5SHhi) (30-04-2014).
Zapata-Ros, M. (2013). MOOC, una visión crítica y una alternativa complementaria: la individualización del aprendizaje y de la ayuda pedagógica. Campus Virtuales, 1, 2, 20-38. (http://goo.gl/RRtaoN) (30-04-2014).