Volume index - Journal index - Article index - Map ---- Back

Comunicar Journal 44: Moocs in Education (Vol. 22 - 2015)

Challenges in the creation, development and implementation of MOOCs: Web Science course at the university of Southampton


Maria-del-Mar Sánchez-Vera

Manuel León-Urrutia

Hugh Davis


Massive is one of the distinctive features of MOOCs which differentiate them from other e-learning experiences. This massiveness entails certain possibilities, but also some challenges that must be taken into consideration when designing and implementing a Massive Open Online Course, in relation to context, work progress, learning activities, assessment, and feedback. This document presents an analysis of the advantages and disadvantages of the massive aspect of MOOCs, and specifically it narrates the experience of creating a MOOC on Web Science, developed at the University of Southampton (United Kingdom) using the new FutureLearn platform, in autumn 2013. In this document, the importance of Web Science as an emerging field is analyzed and its origins explored. The experience gained from the decisions and the work progress developed for the creation and implementation of a specific MOOC is also shared here. The final section of the paper analyses some data from the MOOC in Web Science, including the participation index, the comments and interactions of some participants, tools used, and the organization of facilitation. Challenges involved in running a MOOC related to course design, platform use and course facilitation are also discussed.


MOOC, e-learning, facilitation, technology, university, course, Web Science, resources

PDF file in Spanish

PDF file in English

1. Introduction

Massive Open Online Courses (MOOCs) are so far generating more questions than answers in academia. Such questions are often focused on whether they will be viable in the future, why they generate so much interest, and whether they will transform the future of online education. One of the main reasons so many questions have been asked may be found in the fact that such a phenomenon originated as recently as in 2008, when Siemens and Downes conferred a «massive» feature to open online courses. Three years later, Thrun and Norvig developed an Artificial Intelligence MOOC in which more than 160,000 learners signed up, intensifying scrutiny of the phenomenon and its implications.

Martínez-Abad & al. (2014) analysed the impact of the MOOC acronym, in comparison with the word e-Learning, and found that scientific interest in MOOCs is currently central, with a significantly rising rate of publications. Most of them, however, are more informative than scientific, probably because such a phenomenon is still recent.

Similar conclusions have been drawn from a previous analysis by the British DBIS (2013), in which a steep increase growing curve has been noticed in the rate of publications on the topic, as well as a growing presence of debates on this matter both in the press and social media.

Projects such as OpenupEd and ECO (http://ecolearning.eu), both fostered by the European Commission, reveal the growing interest that European universities are currently placing in promoting online free education (Scopeo, 2013).

Such a trend has also been noticed by Yuan and Powell (2013), who claim the phenomenon is extending worldwide. This article attempts to share the experience of the development process of the first University of Southampton MOOC (UK). The course is entitled «Web Science, How the Web is Changing the World» and it was developed and delivered through a MOOC platform called FutureLearn.

1.1. The challenges of «Massive» in MOOCs

The «massive» nature of MOOCs is perhaps their most distinctive features when compared to other online learning experiences. As Siemens indicates (2012), this «massive» feature became widely discussed when he and Robert Downes delivered a course in which more than 2300 learners subscribed.

Such a high number of learners in a course can contribute to a more effective learning process in several ways:

• Interaction with other learners. This is one of the traditional features of online learning that MOOCs can enhance significantly. A wider network of learners increases the chances of the creation of enriching connections with others worldwide. As the Scopeo June report suggests (2013), MOOCs afford connections with like-minded individuals with similar interests and professional profiles. New groups can be created from these connections, which can generate new ideas for new projects.

• Enhancing the visibility of an institution. One of the main motivations for universities worldwide in designing and implementing MOOCs is that these can become a powerful marketing tool for potential student recruitment.

• Rethinking the curriculum. As Yuan & Powell indicate (2013), MOOC’ popularity could lead HEIs to rethink the elaboration process of the curriculum toward more open and flexible educational models, due to the new «massive» element of these courses.

It is worth noticing that there are certain risks involved for institutions when attempting the creation of such courses, especially if they do not satisfy the innovation and quality requirements set by such institutions.

• The invasion of «package content» The DBIS report on MOOCs (2013) identifies criticism indicating that the spread of MOOCs involves the risk of reproducing online educational models based on «package content» which were common in the 1990s. That is, the emphasis was diverted towards digital resources and their contents, rather than on the educational model and its improvement. Extensive efforts have since been made for more flexible online education that focuses on the process rather than on the product in an attempt to move to a more learner-centric approach. This is why content-centric MOOCs such as xMOOC could set back the progress made in pedagogies underpinning online teaching and learning.

• Assessment difficulties. Because of the high numbers of learners involved, the preference for quiz-like assessments could become a growing trend. Peer assessment as a more flexible option has been practiced for years in contexts such as connectivist MOOCs. However, flaws in this strategy have also been suggested because, as O’Toole indicates, learners are usually provided with templates for grading their peers. Therefore, what is called «peer assessment» should often more accurately be called «peer-grading». A more process-focused assessment is still a major challenge when dealing with such high numbers of students.

• Facilitation challenges. Managing the facilitation of an online course with thousands of learners is far from simple. Personalised feedback becomes complicated when there is a high diversity in tools and approaches used in such populous learning communities (Prendes & Sánchez, 2014).

As discussed above, massive registration is a MOOC feature, but it is not the only one. Low retention rates are also characteristic. Clow (2013) uses the analogy of «the funnel of participation» to explain the process of loss of students from registration to graduation, the latter having rates of between 5 and 15% (Jordan, 2013; UTHSC, 2013; Daradounis & al., 2013).

1.2. A Web Science MOOC

«Web Science» is a growing field of study in the UK. The University of Southampton offers a Bachelors Degree, a Masters of Science, and a doctoral programme in this area. In November 2013, the «Institute of Web Science» was launched with the aim of fostering interdisciplinary research in this area. Its curriculum focuses on the impact of the Web in all areas of society, and it approaches disciplines such as sociology, economy, law, and computer science in an attempt to understand the Web and how it is changing the world. The University of Southampton Web Science Web site (www.southampton.ac.uk/Webscience) presents the subject as a new discipline that has the objective of promoting understanding of what the Web implies as a sociotechnical phenomenon. Tim Berners-Lee, considered the inventor of the World Wide Web, contributed to the establishment of this discipline and its foundations, recommending the identification of needs and changes that the Web is producing in society. The Web, he asserts, should be studied as a social, communicative, and even philosophical phenomenon (Berners-Lee & al., 2006). In this context, the department of Electronics and Computer Science (ECS) of the University of Southampton, together with the above-mentioned Web Science Institute, and the Centre for Innovation and Technologies in Education (CITE), launched a Web Science MOOC that went live on the 18th September 2013 (Davis & al. 2014). It is not a coincidence that the first MOOC produced by this institution is on Web Science, given the prevalence that this field of study is gaining in the institutional agenda of this university (www.southampton.ac.uk/wsi).

Regarding its syllabus and content, the Web Science MOOC is organised over 6 weeks, as shown in table 1.

1.3. Web Science MOOC in FutureLearn

Futurelearn is a private initiative from the Open University in the UK. It operates in a consortium of about 30 institutions, most of them British universities pertaining to the so-called Russell Group. It is a demanding platform in terms of the quality of the materials that it hosts, both pedagogically and technically.

Regarding its pedagogic features, the platform has been inspired on Laurillard’s Conversational Framework (2002), a constructivist model that divides the learning process in four stages (discursive, interactive, adaptive, and reflexive). In each of them, the application of learning technologies can play a fundamental role. Although this is attained to a great extent in Futurelearn MOOCs, their structure still contains certain behaviourist elements related to xMOOC, such as the content sequencing and the quiz-like assessments.

Perhaps the main difference in respect to other platforms is the forum distribution. There is a different discussion forum for each of the course steps, be it a video, an article, or an activity. This way, the discussion threads are not created by the user, but by the educators in the platform.

In order to promote interaction between users, the platform has enabled a system by which they can follow each other, reply comments, vote them (only with positive votes), and rank them in terms of number of votes.

Regarding its assessment system, the platform has recently incorporated a peer review tool, adding it to the existing quizzes.

Another distinctive feature is its user interface, oriented towards a simple and intuitive navigation in order to arrive to a wider target audience. The interface allows the user be aware at all times about their progress by indicating the week in which learners are supposed to be, the week where they actually are, the steps they have completed, and the steps they still have to complete.

For the Web Science course, users were encouraged to use other social media such as Twitter and Google +, although not as the main means of interaction, but as a complement.

2. Method and materials

This article was written by the end of the second edition of the first MOOC at the University of Southampton. It was a six weeks course with a new platform (Futurelearn), and in a relatively new field of study (Web science). Due to this novelty, in many aspects, the education team was unable to predict the course outcomes. The intention here is sharing the experience when dealing with the unknown, present the results obtained so far, and explain how the course was created. Rather than understanding MOOCs as a general phenomenon, it is intended to present what has been classified as an intrinsic case study (Stake, 1994; Buendía & al., 1998).

From its creation to its deployment, the academic and educational team in charge of this project development has divided the work in the following stages:

• Content creation and development. More than 25 staff members of the University were involved in this process, from the dean of the faculty, Wendy Hall, to PhD students. Materials consisted primarily of videos and articles, although some applications and animations were also incorporated. The videos were recorded with TV production means, and hosted in iPlayer, a video platform that comes from the BBC. In fact, Simon Nelson, the production chief executive of Futurelearn, is a former member of the BBC, and responsible for this format. A relatively high budget was dedicated to the video production, especially compared to that of other MOOC platforms.

• The texts and activities proposed by the academic staff were subject to various control processes before being published. One of the main criteria was that these materials had to be succinct, easy to read on screen, and with a plain language that could be easily understood by non-native speakers. Some external articles and videos were recommended for further study, which involved certain challenges with the copyrights. To address this, the library services of the university helped and advised about the legal issues arising from the release of some of the contents.

• The delivery, the facilitation, and the assessment. One week before the course went life, all materials were ready, although there was some work to be done with the assessment. This is an important part of the interaction between the university and the students, and only a few days before the start of the course, the university came to realise that the only form of assessment available was quiz questions (it was in the second run on the course when peer-review was incorporated as an assessment option in the platform). Formulating the right questions involved an extra effort for the educational team, especially due to the presumed diversity of the learning community. Every question had five options, and each of these options contained feedback, regardless of whether they were correct or not.

3. Analysis and results

The data provided by Futurelearn shows that, from the 13.680 registered users, slightly less than half of them (5487) completed at least one step. Nearly 3000 completed steps in more than one week, which suggests that less than a quarter continued to the second week. A survey conducted by the platform, with 802 participants, shows that the main obstacle for completing the course was lack of time, which coincides with the fact that a small majority of participants were working full time (45%).

It is also worth noticing that the highest proportion of participants were over 46 years old (about 20% between 46 and 55, a similar percentage between 56 and 65, and almost 15% more than 66). Also, a majority held a degree (43%), and nearly a quarter had postgraduate qualifications. More than 40% had participated in an online course before. Regarding their professions, education and computing were the two most frequent areas. Regarding their nationalities, three quarters were Europeans, with a predominance of British (63%). Therefore, it could be suggested that the target audience of this course was from the country where the course was developed, mature, with high qualifications, digitally literate, and familiarised with online learning technologies.

In terms of their expectations, participants had «learning new things» as their main motivation (nearly ninety selected this option), followed by the intention to try out the platform as a teaching method (68 participants). Complementing their studies (21) and improving their professional profile (21) appeared not to be the main motivations of the participants. Their interests were mainly related to the area of science and technology (78), followed by humanities (55), and education (43).

1204 learners completed the course, which results in almost 10% of the overall registrars. This situates this course slightly above the average in terms of completing rates, which some studies suggest to be around 7% (Parr, 2013).

Participation in forums was also relatively high compared to other courses in the same platform. The learning community posted around 19.000 comments only in the Futurelearn platform, and 2.200 learners contributed at least once in these forums. More than 1300 contributed at least twice, some 1050 more than 3 times, and about 660 more than four. Graph 1 shows a descending curve in relation to the number of users (vertical axis) in terms of their number of contributions (horizontal axis). It is worth noticing that there were a significant number of users who contributed frequently. Some of them, 7 in total, made more than 100 comments.

Figure 1: users (y) by number of comments (x).

As discussed above, there were different forums for each step. Some of these steps recorded nearly 1000 comments, being the average 151 per forum. As per different weeks, the number of contributions was consistent. Although the first week stood out with nearly 6.500 comments, the subsequent weeks recorded around 2500, except for the last week, with 1.900. The lower figure of last week might be due to the fact that it contained 14 steps, as opposed to the 21 on average of the rest of weeks.

In terms of the nature of the comments, it could be highlighted that most of them consisted of direct responses to questions made within the content of the different steps of the course. For example, there was an activity in which an application used browser history data to return the percentage of web-sites visited by the user. In such activity, users were asked to provide a reflection on their frequency of visits different sites. Most comments in that step (1.425), consisted of the actual reflection.

Comments consisting of a direct question to the educators turned out to be a small minority. Despite that, facilitators replied to an average of 40 comments during the course. It should be taken into account that, as opposed to other MOOCs, facilitators did not post comments for livening up the discussions, but for solving doubts, clarifying concepts, and giving support to issues both technical and content related.

Out of the platform, Google+ was the most utilised space, according to the pre-course and post-course surveys conducted by Futurelearn. The community in this space had nearly 800 members. The number of contributions descended steadily as the course went on, but it kept alive as a space of communication between participants and some educators.

4. Discussion and conclusions: three challenges for the Web Science MOOC

Based on the experience gained from this course and the current literature on the topic, three main challenges can be identified in the creation, delivery and management of a MOOC:

4.1. The course design

The pedagogical design of such a course entailed intense planning and coordination of effort at various levels. The platform was new, so much so that it operated with a beta version at that time. The Web Science MOOC was the first at the University of Southampton, so there was no previous experience to draw from in these kinds of projects. Also, the Web Science Institute is a multidisciplinary department, with subsequent diversity in materials and pedagogical approaches. This situation lead to an enriching process, but it required major efforts in planning and establishing the roles of each contributor, something to be considered in future editions of the course.

How the course will be delivered and what interactions will take place are essential considerations for the design of the course. Yang & al. (2013) suggest that social relationships have an influence in the completion rate of the course. Therefore, as Bentley & al. cofirm (2014) the social side of the course is of paramount importance for its success, and a design oriented to this end should be created so that learners are motivated to participate in communities formed on these courses.

4.2. The platform requirements

There are many reasons why it is considered convenient to use the services of a platform when developing a MOOC. One of them is visibility, a determining factor that both Edinburgh (2013) and University of London (2013) reports recognise as the main reason for joining Coursera. Another reason is the technological support that they offer. Creating a platform for managing the content of a MOOC may involve a cost that exceeds the budget that many universities allocate to free online learning. Outsourcing these services by using established MOOC platforms is often considered a more affordable option.

However, being part of a platform such as FutureLearn entails certain compromises. For example, the course materials, both written and audiovisual, are subject to demanding quality standards. This elevates the production costs to figures that not all institutions can afford. Another compromise to consider is the distribution of contents and activities, as the platform divides everything into «steps» which are categorised into videos, activities, discussions, and assessments. The course educators need to comply with such a classification, which could conflict with their pedagogical aims at times. The same applies for the assessment, as the only options available are quizzes and peer-assessments the protocol of which only the platform controls. Therefore, a divergence with the pedagogical principles of the platform may require a great deal of creativity and flexibility. It is therefore recommended to combine external social media with the social tools available on the platform.

4.3. The challenges of facilitation

Facilitation is one of the greatest challenges not only in MOOCs, but also in any other online learning experience, as students need continued feedback to support their learning process in a context where high levels of autonomy are required (Sangrá, 2001).

Forums are deemed as important communication and learning tools in MOOCs (Mak & al., 2010). Levels of participation in such forums are often indicators of learners’ level of commitment to the course. These participation levels also indicate the liveliness of the learning communities as well as that of the course in general (McGuire, 2013). With these premises, a team of 10 facilitators was established. These were all PhD students at the Web Science Centre for Doctoral Training who were instructed and coordinated in such a way that they could read all comments in the forums, and provide responses when needed. With an awareness of the importance of facilitation strategies in this kind of courses (Marauri, 2013), the following procedure was implemented: a rota with three daily shifts, including weekends, was devised. In each of these shifts, the facilitator would read all comments and indicate in a form which of them had been replied to, and which of them required attention. In a session prior to the course, a protocol was agreed to determine which kinds of actions were going to be taken in different scenarios. One of the main reasons why such a large team was formed is that each of the steps contained a forum, and all of them encouraged learners to participate. Each of the six modules had an average of 20 steps, which generated 120 different interaction spaces in the platform alone. To this we have to add the interactions that occurred in Twitter and Google+. Although there was not an aim of replying to all the nearly 19,000 comments, the facilitation team aimed to go through all of them in order not to leave unanswered questions or doubts. The intention was also to let learners be the drivers of the conversations. It was observed that in very few occasions these interactions went off topic, perhaps because in the platform structure, the content of each step determined the conversation topic. The challenge is fostering participant interaction, and the creation of conversational threads and groups of learners that interact with each other.

4.4. The challenges of the MOOC phenomenon

The traditional challenge of online education, namely activity design, facilitation, assessment, and feedback (Burkle, 2004; Prendes, 2007; Sánchez-Vera, 2010), prevail and even intensify with MOOCs, especially due to their massive size. However, despite their difficulties, MOOCs open a wide range of possibilities, as they are not only about opening up resources, but about the whole educational process. Thus, these courses represent interesting learning and professional training opportunities, and can even be advantageous for their use in Flipped Classroom experiences (Zhang, 2013).

The experience presented here does not represent the end, but the beginning of a promising path towards the improvement and widening of online learning opportunities.


Bentley, P., Crump, H., Cuffe, P., Gniadek, B.J., MacNeill, S. & Mor, Y. (2014). Signals of Success and Self-Directed Learning. EMOOC 2014: European MOOC Stakeholder Summit. Proceedings, 5-10. (http://goo.gl/jkHP4q) (03-04-2014).

Berners-Lee, T., Hall, W., Hendler, J.A., O´Hara, K., Shadbolt, N. & Weitzner, J. (2006). A Framework for Web Science. Foundations and Trends in Web Science, 1, 1-130. (http://goo.gl/jZ9vl7) (14-04-2014) (DOI: DOI: http://doi.org/dsh2b8)

Buendía, L., Colás. P. & Hernández, F. (1998). Métodos de investigación en psicopedagogía. Madrid: Mc Graw Hill.

Burkle, M. (2004). El aprendizaje on-line: oportunidades y retos en instituciones politécnicas. Comunicar, 37, 45-53. (DOI: http://doi.org/fc2tgj).

Clow, D. (2013). MOOC and the Funnel of Participation. Third Conference on Learning Analytics and Knowledge (LAK 2013), 8-12. Leuven. Belgium. (http://goo.gl/KCSqAJ) (20-03-2014) (DOI: http://doi.org/tqz)

Daradounis, T., Bassi, R., Xhafa, F., & Caballé, S. (2013). A Review of Massive e-learning (MOOC) Design, Delivery and Assessment. Eight International Conference on P2P. (http://goo.gl/9QKrtr) (21-03-2014) (DOI: http://doi.org/tpk).

Davis, H., Dickens, K., León, M., Sánchez-Vera, M.M. & White, S. (2014). MOOC for Universities and Learners an Analysis of Motivating Factors. 6th International Conference on Computer Supported Education.

Department for Business, Innovation & Skills (2013). The Maturing of the MOOC: Literature Review of Massive Open Online Courses and Other Forms of Online Distance Learning. United Kingdom. (http://goo.gl/X8UlG4) (16-04-2014).

Jordan, K. (2013). Synthesising MOOC Completion Rates. MoocMoocher. (http://goo.gl/8yyu6r) (14-03-2014).

Laurillard, D. (2002). Rethinking University Teaching: A Conversational Framework for the Effective Use of Learning Technologies. London: Routledge Falmer.

Mak, S., Williams, R. & Mackness, J. (2010). Blogs and Forums as Communication and Learning Tools in a MOOC. Networked Learing Conference, 275-285. University of Lancaster.

Marauri, P.M. (2013). Figura de los facilitadores en los cursos online masivos y abiertos (COMA/MOOC): nuevo rol profesional para los entornos educativos en abierto. RIED, 17, 1, 35-67. (DOI: http://doi.org/tq2).

Martínez-Abad, F., Rodríguez-Conde, M.J. & García-Peñalver, F.J. (2014). Evaluación del impacto del término MOOC vs eLearning en la literatura científica y de divulgación. Profesorado, 18, 1, 1-17.

McGuire, R. (2013). Building a Sense of Community in MOOC. Campus Technology. (http://goo.gl/dcS3ls) (27-03-2014).

O´Toole, R. (2013). Pedagogical Strategies and Technologies for Peer Assessment in Massively Open Online Courses (MOOC). University of Warwick. (http://goo.gl/16rlIF) (20-04-2014).

Parr, C. (2013). MOOC Completion Rates Below. Times Higher Education. (http://goo.gl/pQBKls) (20-04-2014).

Prendes, M.P. & Sánchez-Vera, M.M. (2014). Arquímedes y la tecnología educativa: un análisis crítico en torno a los MOOC. REIFOP, 79.

Prendes, M.P. (2007). Internet aplicado a la educación: estrategias didácticas y metodologías. In J. Cabero, (Coord.), Las nuevas tecnologías aplicadas a la educación. Madrid: McGrawHill.

Sánchez-Vera, M.M. (2010). Espacios virtuales para la evaluación de aprendizajes basados en herramientas de Web Semántica. Universidad de Murcia: Tesis doctoral inédita.

Sangrá, A. (2001). La calidad en las experiencias virtuales de Educación Superior. Conferencia Internacional sobre Educación Superior, Formación y Nuevas Tecnologías, 641-625.

SCOPEO. (2013). MOOC?: Estado de la situación actual, posibilidades, retos y futuro. Scopeo, Informe, 2, 266. (http://goo.gl/rVD7tR) (18-04-2014).

Siemens, G. (2012). What is the Theory that Underpin our MOOC? (http://goo.gl/nHhCOJ) (25-03-2014).

Stake, R. (1994). Case studies, In N.K. Denzi & Y.S. Lincoln (Eds.), Handbook of Qualitative Research. Thousand Oaks: Sage Publications.

University of London (2013). Massive Online Open Courses (MOOC) Report. (pp. 1-40). London: The University of London. (http://goo.gl/RJCbo4) (24-04-2014).

UTHS (2013). What is a MOOC. UTHS Educational Technology. (http://goo.gl/djRNty) (03-04-2014).

Yang, D., Shina, T., Adamson, D. & Rosa, C.P. (2013). Turn On, Tune In, Drop Out: Anticipating Students Dropouts in Massive Open Online Courses. Proceedings of the 2013 NIPS Data-Driven Education Workshop. (http://goo.gl/t2qtIm) (14-03-2014).

Yuan, L., & Powell, S. (2013). MOOC and Open Education. A White Paper, 1-21. (http://goo.gl/Yw2CvV) (16-04-2014).

Zhang, Y. (2013). Benefiting from MOOC. In A. Herrington, V. Couros & V. Irvine (Eds.), World Conference on Educational Multimedia, Hypermedia and Telecommunications, 2013, 1372-1377. AACE. (http://goo.gl/Q3pXhZ) (20-04-2014).