Volume index - Journal index - Article index - Map ---- Back


Comunicar Journal 35: Film Languages in the European Collective Memory (Vol. 18 - 2010)

Discourse analysis in the quality of expected learning

https://doi.org/10.3916/C35-2010-03-06

Ana-Elena Schalk-Quintanar

Carlos Marcelo-García

Abstract

Universities are gradually implementing virtual learning processes. However, research still remains limited in examining the internal processes that occur in learning in virtual environments. This article presents an investigation that seeks to describe the relationship between the quality of interaction in asynchronous discussion forums in training experiences in e-learning, and the quality of learning offered and achieved. The main objective was to determine how interactions in online environments add quality to the learning of students. For this, a descriptive investigation was done that combines qualitative and quantitative phase, analyzing more than 10,000 messages in 171 participants from four postgraduate courses developed in the form of e-learning. Asynchronous communication was analyzed through a category system that analyzes the social, cognitive and didactic discourse online. Among the research findings, there highlights a positive relationship between quality and quantity of speech of the participants and the quality of learning achieved and reflected in the different levels of assessment. We can conclude that the need to analyze, not only the written discourse in asynchronous communication, but also to establish relations with both cognitive and social learning of students. Moreover, we conclude the necessity to train teachers to deal with the processes of online communication.

Keywords

E-learning, quality learning, interaction, asynchronous communication, on-line education, knowledge building

PDF file in Spanish

PDF file in English

1. Introduction

This research adds to the line of work, which since the late 90’s, began to analyze and assess the relevance of computer-mediated education. From Mason (1990) there has been offered a framework for understanding computer-mediated communication. It has been a distinction between synchronous and asynchronous communication. Research proposals like those of Van Dijk (2000) and Shotsberger (2001) have explored discourse analysis from different points of view. This helped to understand that it is not the amount of interaction but the quality of them, which allows us to investigate and try to understand how the learning process occurs through the interaction and exchange of ideas in computer-mediated communication (Cebrián, 2009) .

Gunawardena and his colleagues (1997) undertook the task of defining a model that through the instrument could be used to examine the construction of knowledge. They are based on a grounded theory and use their stages of discussion to determine the weight of knowledge built. This analytical model offers important elements to understand the construction process, both for teaching and learning in collaborative environments, since it is centered on the interaction as a vehicle for building knowledge, it detects the knowledge building that arises in a conference; it is very appropriate to consider the context of learning and has a relative strength in its framework. In 1999, Rourke and others identified three elements for the community of inquiry. The other two were the cognitive presence and the teaching presence. They stressed the importance of social presence to motivate students in their learning process. This social dimension is configured in three categories: emotional responses, interactive responses and responses of cohesion.

Later on, Garrison, Anderson and Archer (2001) identified the cognitive presence in the community of the inquiry model. Their presence reflects higher-order knowledge and an application that is usually acquired based on the literature and research related to critical thinking. They worked out four phases: initiation, exploration, integration and resolution. They felt that the complete message would be the unit of analysis of their work. It was tested in two separate studies and their reliability bases were measured by the rate of Holsti and of Kappa, obtaining good level results at both.

At the same time, Anderson and others (2001) developed a proposal to analyze the presence of teaching under the framework of communities of inquiry. They considered these major roles: experience design, the facilitation and co-creation for the conduct of an active social environment, the mastery of subject that would allow students to have a direct instruction. Its reliability was tested by the bases of Kappa obtaining a high level of consistency. The work of Duffy and Jonassen (1992), Hillman (1994), Bonk and King (1998), Paloff, (1999), the OECD (2001) as well as the undeniable contributions of Garrison and Anderson (2004) have developed a complete line of research focusing on the process of teaching and learning in this modality. All of them provide the background underlying this work.

This study follows the line of production of knowledge covered by researchers such as Marcelo (2002), Marcelo and Perera (2002) and Perera (2007). In this specific case, the paper boards its analysis, linking three relevant elements that interact in a virtual learning experience. These are: quantity and quality of interactions, results of the learning units and quality of final work that should account for the implementation of these learning outcomes.

Main Question: is there a relationship between quantity and quality of participation and interactions that occur in asynchronous discussion forums and the expected quality of learning in e-learning experiences selected? In what way is it happening? The centrality of this work concerns the search for evidence intending that it can relate the two vectors, quantity and quality of interactions and the achievement of expected learning (reflected in the evaluations of each module, in its self-evaluation and its final work), to identify it all, common elements like those that are differentiating, and that allow to obtain relevant information to enhance the design, execution and evaluation of the future educational activities in e-learning.

On one hand, there is enough evidence regarding the analysis of discourse in asynchronous communication forums, and on the other hand there is a varied number of studies that address the learning experience from different angles: the model of design, didactics, etc. However, in the studies reviewed for this article, there is always a challenge present to inquire more about: how do students learn through the forum? In what way does learning occur in virtual courses and how does it relate to the activities in the forums? How can we enhance the value of building knowledge and learning with others by e-learning? (Schrire, 2006; Fainholc, 2006, De Wever, 2006; Perera, 2007).

2. Material, methods and sample

This research was carried out through a descriptive study to investigate the presence and type of relationship between two variables: the variable participation/interaction in asynchronous discussion forums, and the variable quality of learning expected. This variable is reflected in: qualifications of the learning modules and quality of the final project or work where they could identify the applications and transfers of these through its implementation.

We analyzed two variables: analysis of participation / interaction and quality of learning expected in a kind of descriptive research that seeks to explain whether occurring and how it is established the relationship between these variables in virtual learning experiences of similar level. The analysis is performed through two-dimensional paradigms using qualitative (discourse analysis, analysis of the quality of learning expected) and quantity (frequency of intervention and achievement by learning modules). After this analysis, four cases are set to deepen maximum variability in the description that explains the relationship between variables that must comply with: 1) scores on the learning modules, 2) discourse in the discussion forums, 3) quality of discourse (discourse analysis), 4) quality of learning in observable actions (final working drawing), 5) self-assessment (post training).

Three main reasons motivated the decision to take the model of discourse analysis as proposed by the team of Anderson, Garrison, Archer, and Rourke from the University of Alberta (Canada) (Anderson, 2001; Garrison, 2003.)

The first reason, has to do with informed analysis based on three criteria established as standards of quality of available models (De Wever, 2006) and the results observed by researchers, the solid theoretical framework of this model, the solid arguments of the same, the considerations relating to the defined unit of analysis, and finally the good results of reliability were the main elements to consider in model selection.

The second with the background work that the authors had done on this model and the last in the interest of continuing the line of research generated through this IDEA group scheme, which has its roots since 2004 and adds to this line of research a different vision or look from those been considered so far in this work.

The three dimensions that constitute this model are: the cognitive dimension, the social and didactic dimension. Each one contains subcategories of the structured analysis as follows:

The cognitive dimension looks to identify through sustained dialogue on the forums, discourse units which reflect the capacity of participants to develop, build and express their thoughts.

For this dimension, the initiation of dialogue, the search for information or ideas that could favor the solution (if it is a problem) or the possibility of new ways to resolve the situations they face in creative and innovative processes, is the reason for focalizing this dimension. The interaction in the teaching-learning process could not be conceivable without the presence of this dimension, since it is precisely where the manifestation lies in the thought process and the building students transmit through language, expressed in this case, in their interventions realized in the discussion forums.

The social dimension is a fundamental element in this system of categories of discourse analysis, because it allows identifying those elements on the expression of feeling of the participants. In this dimension situations are valued where «the person» is expressed as such and therefore offers an opportunity for a relationship beyond cognition, where the feelings are involved and there is set up a social space for learning to give cohesion and strength in the working group.

Finally, the didactic dimension, which is focused on results obtained in the cognitive process and social interaction. This dimension permits to identify situations where they expose new questions, react to the interventions of others, the responses are scaled to sort and to synthesize a common conclusion. Natural to the process of teaching and learning, this dimension detects, organizes and systematizes all the evidence offered by the speech held at the forum, to consolidate learning expected in students.

3. Sample:

Between 2005 and 2007 research was undertaken to analyze the expected learning participation as an object of study taking courses in e-learning at the University of Sevilla (Schalk, 2007). Based on this argument, we specify the sample to this research in: Analysis Group 1 (version since October 2005 to June 2006); Analysis Group 2 (version since October 2006 to June 2007); Analysis Group 3 (version since October 2006 to June 2007) and Analysis Group 4 (version Master degree since October 2006 to June 2007).The academic certification was an Expert Level for the groups 1, 2 and 3, and its duration was 280 formation hours. The Master degree included 340 training hours taught in two years. The first year was the Expert level.. The total number of participants are 171: students, invited teachers, tutors and director. All of them were distributed in the following way: Group 1 (Expert 2005) had 65 people. Group 2, had 51 participants; Group 3, 31 participants; and Group 4 (Master Degree Group) 26 participants.The sample selected to analyze the discussion forums were all of them where the students and teachers (including tutors like as) interacted. For this reason the sample was of 55 asynchrony communication forums interaction generated 10,299 messages unit analyzed made by 171 participants. For cases (those of maximum variability) the fulfillment of the following requirements was considered: dimensions of discourse found in each participant; dynamics of such participation / interaction / interactivity (map of interventions); to have all their evaluations of the modules, to have the 13 criteria for assessment of their final work and to have their answers to the self-assessment instrument.

4. Results

E-learning is a form of computer-mediated learning, which is based on interactivity, and this is facilitated through the design and implementation of experiences based on constructivist theory (individual and social), through the formation of communities of inquiry-all for the development of critical thinking that enable better and higher quality of learning outcomes. In turn, this can be analyzed through discourse and interaction in the areas of communication (in this study, referring to asynchronous communication). Analyzing it is complex and multifaceted.Therefore, to understand their relationship and impact on learning is not affordable in a linear fashion, so:

• How are distributed contributions of students in the forums of the courses chosen, according to the profile of the actors?

In all courses included in this study, we can demonstrate that participation of the tutors in the activity of interaction in the asynchronous communication spaces is about 30% and that the primary interaction focused on the students (70%) and therefore it can be said that it is mainly them who are the agents. The interventions of the tutors in all these experiences did not exceed 100 per module, even when the dynamics of interaction between students was significantly different in the three expert-level experiences. Along the same lines and in almost all cases analyzed, the more increased activity of students, the greater involvement of the tutors. That is, in presenting these results we can establish a bidirectional link between the activity of the tutor for the facilitation of speech and activity of the students that energizes the tutors involvement.

An important aspect is that when people or styles are combined in one same group or version of a course, the dynamics of participation is higher because of the nature of the people who agree on a course, so much so that even the questions and the most technical aspects, that generally tend to clarify the chain of communication, allow in the same way, an active relationship between students. However, although this aspect needs to be considered, so far, in the same way as in the present education, it is a very hazardous profile of people converging on a training activity. That is, it directly impacts the dynamics of participation, but this situation is not controllable.

• Does the nature of the learning content directly influence the quantity and quality of participation and interactions?

Through the obtained results it can be shown that there exists a direct relationship between the amount of participation and the type of learning content and that when the content is procedural in nature students tend to go to the forum for specific questions and get answers almost without any unequivocal evidence that might address differently the same procedure.

• Is it possible to establish a relationship between the amount of participations in the forums and the evaluation results obtained by the students?

The evidence found in the experiences of three of the four activities selected for the study, is that through the means of participation and the grades expected in learning in each of the training modules, there is a very large variability in participation rates and yet, it appears that the learning achieved, reflected in average scores for the modules are between a 6 and 10 rating except in two occasions where both average ratings corresponded to the lowest participation averages. However the relationship between the participation and the learning expected in each module was of a high variability and what can be inferred by contrasting the performance of all of the averages of both variables is that, the highest level of participation reached the highest level grade and the lowest level of participation was the lowest level grade which allows to conclude that there exists a positive relationship between both variables.

• Is there a relationship between the amount of participating and the evaluation of the expected learning assessment?, Which?

According to the grounded theory that holds the model of discourse analysis in this study, we propose that for learning to occur in the e-learning there should be a significant relationship between two variables. That is, it is expected that in a virtual environment, people «learn more and better if they interact actively in their learning experience with others. However, this study shows that students can learn and are able to «do» or perform important procedures, without necessarily requiring to be with others, build with others or learn from others.

• How can we set the participation of students and their distribution according to the structure of categories of analysis chosen?

Delving into discourse analysis, we conclude that the experiences chosen contain a high level of social dimension, as it is the most concentrated in the speech frequencies. After this dimension the didactic one, although only for the 2005 version, was higher than the social one, but not significantly, but for other courses it would be to follows this educational dimension in the presence and frequency. In the end, the cognitive dimension is set, which is significantly lower in presence of all courses. A frequency behaviour of the different dimensions can be noticed that is consistent between groups. In all of them there is a significantly prevalent social and didactic presence, while the cognitive dimension stays of less importance.

This level of analysis leads to formulate what the study of Perera, (2007) concluded: the need to strengthen the processes of collaborative learning where the tutor can enhance personal communication and promote a sense of community learning, where activities may generate or devise suppelemntary activities that allow to construct knowledge together with the course content, giving the opportunity to go to different students to make commitments to revitalize the forums, it is necessary to promote a high degree of interactivity among students, not so much the participation of individual responses, but in the generation of common responses which invites students to look from different perspectives at the problem or content and also to ensure that participation is an opportunity for interactivity by encouraging the development of critical and creative thinking. And that, definitely lies with the tutor’s role as a model of interaction. Such interventions result in mobilization of discourses as proposed by Lipman (1991) more than a decade ago: soliciting arguments that apply to the participation, looking for examples, giving counter-examples, doing exercises of evaluation and weight of the arguments, looking for applications and validating generalizations, detecting false generalizations, analyzing the part-whole relation and vice versa, etc.

In this sense it is concluded that there does exist a more didactic presence and cognitive modeling by tutors, the students are also motivated and find a sense of interaction as part of extended learning. Let’s make it clear, that this study does not devalue the contribution that brings the social dimension in the process of interaction and virtual learning, so it is argued in conclusion, that there must be a balance, with increasing trend in presence of the other two dimensions which may favor the social construction of knowledge and there may be a better relationship between the quality of learning and interaction.

The previous argues against the obtained results in which students report a high level and more permanent presence and frequency in all courses and all dimensions analyzed. If the tutor facilitates, guides, promotes and participates in search and social construction of learning where critical and creative thinking are manifested through the exchange achieved in an environment of «community of inquiry or learning» then, students have the pedagogical conditions needed to continue a constructive dialogue, production and development in the quality of their learning.

The tutors reported a weak and very low cognitive presence unlike students who show an ongoing activity in this dimension, even if this dimension is where they could be expected to guide, stimulate, shape, and take the necessary decisions to raise the quality of the discourse and interaction in terms of the learning expected.

However, to establish or conclude that the responsibility of interaction rests entirely on the tutors would be wrong. If you expect students to be active participants in a community of inquiry through the forums and keep an interactive dialogue, it is necessary to provide an educational effort intended to develop basic skills of written communication which makes it more complex, which itself requires an effort of teaching for the development of a community of inquiry for critical and reflective thinking.

The most recurrent themes in the speech content of the forums can be seen in the diagram of categories.

In the forums of learning modules, there are two large groups of subjects identified: those associated with dialogues that deal with the content itself, (which are almost always present in modules 1-6 for all courses and modules 7 and 8 of the product B) and those that apply to the use, techniques or ways of doing or incorporating the use of tools (both routine technology and in the education) that are more significantly present in the modules of the product A and in modules 9 in front of the product B. In the case of the Master, the modules related to declarative learning content, concepts and ideas (for example, the introduction to SCORM standards) belong to the first group of this analysis, and modules referring to «procedures» (with the same example, create and implement data and metadata of the created virtual material) would be contained in the second group of topics related to how to do things.

As noted in the analysis of the previous results, a direct relationship is found between the type of content to learn and the quality and form of interaction that develops between the participants, so that the themes that are geared towards the use of technology tools was lower than the other subjects, and also had a speech quality with less presence of social and cognitive dimensions and more toward teaching and specific questions to solve problems associated with the use of it (where it was expected that responses would fall more in the tutorial).

5. Discussion

On the planning and design of a course, the studies of Hara and others (2000) analyze how technologies and the ALN (Asynchronous Learning Networks) can support the development of higher-order cognitive functions, transforming education, creating environments more focused on the students to interact with peers (critical thinking). Moreover, they suggest that ALN support constructivist learning because they allow students to articulate, read and reflect on concepts, as well as deferring this communication allows students to have control over the reflection. Therefore, these aspects must remain present in the forming of structure, design and development of virtual training activities. This constructivism must intentionally develop activities, learning goals and facilities to promote interaction and social construction of knowledge (Jiménez & Llitjós, 2006).

It is suggested that training in e-learning integrates different learning styles, which currently and increasingly, technology makes possible, to develop pedagogical models aimed at developing multiple intelligences, to build knowledge through various channels of information processing, to promote the use of collaborative tools that the Web 2.0 already has available, to open web spaces and to encourage the intra-network works, making the interaction meaningful, understandable and valuable to students in terms of what they need to learn.

Another option, when planning, is how to form groups of convergence (training, of initial competitions of interest, by choice) to enable effective discussion and exchange of ideas that are constructive for the members but combining it with other spaces «virtual community» where all participants are integrated into a common interaction and reducing the risk of being permanently beginning the process of virtual community for each of these groups that are formed for the work of learning.

In this aspect where the interaction is a direct element of learning in e-learning, we think of the questions we should ask and rethink when designing an e-learning activity: how will we take care of those who come to these experiences without this referred competition «technique»? How will we make them learn in the same way than the others? How will we detect them, support them and facilitate their development?

In the field or level of implementation and specifically on the tutor mentoring, as seen in this and other studies, it is necessary to facilitate the approach to content and information, where it focuses on the learning process (Brown, 2003). However, there are other factors that influence the virtual interaction, and that should be considered to facilitate and participate in this process.

These elements or features are as those suggested by Pallof and Pratt (1999) the ability to provide a speech that builds knowledge, that develops cognitive and creative skills in students, that provides the ability to share the learning, to negotiate, to solve problems and to raise new visions of a problem, to be part of a community of inquiry, and especially to be able to be an element of a «builder» of that community.

The close relationship between the individual and the collective world (or shared) is the difference which falls between a mediated learning experience and the technology based on the exchange, cooperation and reconstruction of knowledge starting from the interaction. On this basis, it is important to consider that the interaction between teachers and students cannot be analyzed separately and thus the process of interaction should be viewed in a unified manner. The implications of these principles of the theory result in important implications for understanding how learning occurs in e-learning.

If the interaction is a unified process that is permanently defined, orientated and transformed according to what the tutors and students do, then while enquiring how learning occurs and the impact of interaction on it, it reveals a need not only to analyze student activity (which is expected to obtain the learning awaited) but also what happened with the tutors to produce them. And this study can offer a well-defined explanation of how this process occurs and how to strengthen the presence of a more reflective and critical discourse that enables the consolidation of a community of inquiry that promotes social interaction, but that projects towards learning levels at a greater scope for students. With regards to the evaluation, the student and the tutor or teacher should establish criteria based on assessment feedback, collaborative learning, on the self-evaluation of active learning, where the role of students in the process of social construction of knowledge is considered as an indicator of quality standards.

References

Anderson, G. (2004). E-Learning in the 21st Century: A Framework for Research and Practice. London: Routledge Falmer.

Anderson, T.; Rourke, L. & Garrison, D. (2001). Assessing Teaching Presence in Asynchronous Text-Based Computer Conferencing, 5; (2-3).

Castaño, C. (2003). El rol del profesor en la transición de la enseñanza presencial al aprendizaje on-line. Comunicar, 21; 49-55.

Cebrián Herreros, M. (2009). Comunicación interactiva en los cibermedios. Comunicar, 33; 15-24.

De Wever, B.; Schellens, T.; Valcke, M. & Van Keer, H. (2006). Content Analysis Schemes to Analyze Transcripts of On-line Asynchronous Discussion Groups: A Review. London: Science Direct, Elsevier; 46; 6-28.

Duffy, T. & Jonassen, D. (1992). Constructivism and the Techonology of Instruction. London:United States of America: LEA Editors.

Fainholc, B. (2006). La interactividad en la educación a distancia. Buenos Aires: Paidós.

Garrison, D. & Anderson, T. (2003). E-learning in the 21th Century. Great Britain: RoutledgeFalmer.

Garrison, D.R.; Anderson, T. & Archer, W. (2001): Critical Thinking, Cognitive Presence, and Computer Conferencing in Distance Education. American Journal of Distance Education, 15 (1); 7-23.

Hara, N.; Bonk, C. & Angeli, C. (2000): Content Analysis of On-line Discussion in an Applied Educational Psychology Course. Instructional Science, 28; 115-152.

Hillman, W. & Gunawardena, A. (1994). Learner-interface Interaction in Distance Education: an Extension of Contemporary Models and Strategies for Practitioners. American Journal of Distance Education, 8; 30-42.

Jiménez, G. & Llitjós, A. (2006). Procesos comunicativos en entornos telemáticos cooperativos. Comunicar, 47; 149-154.

Lipman, M. (1991). Thinking in Education. Cambridge: Cambridge University Press.

Marcelo, C. & Perera, V.H. (2007). Comunicación y aprendizaje electrónico: la interacción didáctica en los nuevos espacios virtuales de aprendizaje. Revista de Educación, 343; 381-429. Mason, R. (1990). Computer Conferencing in Distance Education. En Bates, A. (Ed.). Media and Technology in European Distance Education. London: Routledge Falmer Studies in Distance Education Series.

Marcelo, C. (2002). Aprender con otros en la red. Investigando evidencias. Zaragoza: Congreso Virtual Educa 2002.

OECD. (2001). E-learning: The Partnership Challenge. Paris: OECD.

Palloff, R. & Pratt, K. (1999). Building Learning Communities in Cyberspace. San Francisco, CA: Jossey-Bass Publishers.

Perera, V. (2007). Estudio de la interacción didáctica en e-learning. Sevilla: Universidad de Sevilla.

Rourke, L.; Anderson, L.; Garrison, D. & Archer, W. (2001): Methodological Issues in the Content Analysis of Computer Conference Transcripts. International Journal of Artificial Inteligence in Education, 12; 8-22.

Schalk, A. (2006). Modelo pedagógico para formación de adultos en el desarrollo de competencias. Gestión en el Tercer Milenio. UNMSM, 8, 16; Revista de Investigación de la Facultad de Ciencias Administrativas,. Lima, diciembre 2005).63-87.

Schalk, A.E. (2007). El valor de la comunicación e interacción asincrónica en una experiencia de formación virtual: análisis del discurso, desarrollo de competencias y calidad del producto final con los aprendizajes esperados. Programa de Experto y Máster en E-learning de la Universidad de Sevilla.

Schotsberger, P. (2001). Classifying Forms of Synchronous Dialogue Resulting from Web-Based Teacher Professional Development. Orlando: SITE Conference.

Schrire, S. (2006). Knowledge Building in Asynchronous Discussion Groups: Going Beyond Quantitative Analysis. Computer & Education, 46; 49-70.

Van Dijk, T.A. (2000). El discurso como interacción social. In El discurso como interacción social. Estudios sobre el discurso II: Una introducción multidisciplinaria. Barcelona: Gedisa; 213-262.