Volume index - Journal index - Article index - Map ---- Back
Media literacy, innovative didacticism, design, interactivity, immersion, interactive communication, multimedia
The start of the 21st century has seen a huge increase in the number of hours the youth spend on using new technologies, which has led to numerous studies into why video games are so attractive. The model presented in Epigraph 3.3 can be used and redirected for the transmission of cultural and educational content through new media. Gee (2003: 200-212) underlines the importance of studies from the perspective of learning about the new media elements that young adults find most attractive.
To study the hypermedia interface was the main purpose of this research since it is the main technological and expressive channel that gives access to interactive and multidirectional media communications. New media integrate the qualities of interactivity and multimedia within the digital interfaces. The hypermedia interface is changing the human communication system due to its interactive ability as well as the multisensory qualities of the hypermedia. Each hypermedia element that forms part of the interface is located exactly between the information flow generated by several communicative actors, the original author of the interactive media product and its users, players, students or receptors. According to Osorio & Duart (2010) «interaction can be defined as the cognitive and social actions between the actors of the educative process (student-professor, student-student) in the development of the learning activities». The term interactivity used in this research is also similar to the definition given to it by Meadows (2003: 37-39) «to denote the relationship between the human/s through an object, the computer [...] The interactivity requires the fulfillment of a series of guidelines to have flow».
This approach can also be used to answer important contemporary questions in the area of educommunication and alphabetization with new media. Questions that are important to find answers to, such as: how can we develop immersive interfaces for educational content?, or González-Faraco and Gramigna’s (2009) statement that «by studying both the logic of video games and how players understand them, we can develop interesting ideas for re-thinking theories on knowledge and education».
The interface is defined by the «RAE dictionary», «Diccionario de la Real Academia de la Lengua Española», as «the physical and functional connection between two devices or independent systems». «The New Oxford American Dictionary» describes the interface as «a point where two systems, subjects, organizations, etc, meet and interact». In this sense, the interface is the physical and functional connection between the interactive communicative system, its interactive aesthetic and narrative expressions and the human system of perception and communication. It is Moreno-Muñoz´s (2000: 114) definition that fits best with this research: «the mixture of hardware and software that the reader-author uses to communicate with the hypermedia program». The interface synthesizes the conjunction of interactive expressions and educational content organized within the multi-sensorial representations permitted by informatics. The author-sender of the didactic communication manages the technological possibilities in order to transmit educational representations and narratives. Thus, he will try to create original expressions organized in an attractive multimedia form that can stimulate a new interaction in the student. Then, he becomes a reader-author, receptor-sender, even student-professor committed to the learning process. The main goals and motivation behind this research were to find those principles that add quality and depth to interactions with the hypermedia interface and the edutainment content.
The material and methods used in this research integrate an interdisciplinary approach that results from the complexity of the hypermedia interface. It includes areas of interactive communication and narrative with perception, education and psychology, among others. The following materials observe and describe how the original form of that fusion between communicative expressions and narrative representations occurs thanks to the hypermedia interface. This research was initially conducted by the application of new models of communication and analysis of the hypermedia elements, tested on several educational interfaces over a period of 8 years using discussion groups, surveys, questionnaires, multi-camera observations, content analysis, etc. The models presented have been applied to different educational and cultural interactive media projects, and this paper includes the improved versions of these models. The conclusions of previous research were applied to several projects, like the creation of an online interactive museum (www.multiculturalvideos.org) presented at the Communicability Workshop, ACM Multimedia Conference (Mora, 2008), or the prototype which won awards for a massive cultural multiplayer online game called Exchanging Cultures EC Game. These conclusions were also applied to the design and software selection of the international cooperation educational intervention for hypermedia alphabetization in Peru and Bolivia, within a group of 200 students. Thanks to the application of these models successful interactive educational workshops were developed, and the most immersive digital video editing software was selected for use during the intervention, e.g.: Final Cut, Premiere and Sony Vegas. The implementation of new innovative models of analysis of hypermedia interfaces were developed and tested during this process, and the results are detailed in the epigraph that follows. These models can be used in educational environments, cultural videogames, interactive museums, cinema, webs, digital journalism and augmented reality.
This interactive educational strategy and models were field tested, confirmed and applied in one recent international cooperation experience between USA and Spain, in Peruvian and Bolivian educational and cultural institutions in the cities of Cusco, Puno, La Paz and Oruro, with the participation of approximately 200 students during the summers of 2010 and 2011. As Arcila & Said (2011) underlined «According to the results, a good part of digital media in Latin America are incorporating web 2.0 tools, but there still exist notable quality differences in the ranking proposed between the best rated media (Colombia and Mexico) and the worst rated (Chile and Bolivia)». The cooperative goal was to introduce and educate Peruvian and Bolivian artists and students in audiovisual and multimedia communications, using hypermedia interfaces and software for digital video editing. This way they were able to develop videos for the Internet while audovisually promoting and conserving their Quechua, Aymara and contemporary Andean cultures.
To obtain that goal, several video editing interfaces such as Final Cut, Adobe Premiere and Sony Vegas were offered to the students, and installed on their own computer stations or on those brought by the cooperating organizations. These software programs were the most current versions and were compatible with PC or Mac systems, so every student could find a familiar way to interact and learn through the process. Moreover, we brought the most updated and/or expressive versions, so the students could have hypermedia interfaces to materialize the representations and contents they wanted through easy immersive interactions. The classes were also designed to be interactive, so after a brief and synthesized explanation about writing a script, organizing technical equipment for recording, or about capturing and editing the recorded images, the students could interact with the equipment and the software interfaces. Finally, they were able to apply the lessons, expressing and materializing their representations right away by using the hypermedia interfaces. That way they also helped implement the teaching techniques during the experience, since they were very interactive with the lessons, the tools and the contents they were using. The results were surprising: 10 videos, 10 minutes each, were produced in a week, made by groups of three to seven members, in a class of 60-70 students per city. There were a total of 45 videos, and the experience also allowed the implementation of the model of analysis of effective hypermedia interfaces, presented here in the following epigraph 3.3.
The following systems were applied during the research experience to implement the dynamics of alphabetization in media literacy. The traditional dialectic system of mass media communication developed by Martín-Serrano (1982: 159-174) is presented in Figure 1, and contrasted with the model in Figure 2, which includes the new elements that the hypermedia communication system integrates.
The hypermedia interface can be located in the center of the cross in Figure 2 since it interconnects and interexchanges all the communication elements: actors, expressions, representations and instruments.
In the hypermedia communicative system, the positions and functions of the communicators change over time, when the communicative phenomenon is produced through interactivity. The hypermedia interface allows this exchange of roles since it integrates multi-sensorial expressions with different types of interactions. The actor looks forward to communicating his own representations through the interface, whether, he is communicating with an artificial intelligence (AI) system such as a one-player video game, or with other human actors, such as playing in a massive multiplayer online game (MMOG). Afterwards, the receptor actor decodes mentally and emotionally the representations contained in the multimedia expressions presented through the interface. Then, the differential moment of the interactive communication is when the motivated actor decides to activate the multi-directional system of communication, using his interactions over the hypermedia interface, and coding his own representations to communicate them to a new receptor, a.k.a. reader-author, student-professor, author-reader, or the first sender. The most characteristic moment is when the exchange of roles between actors occurs: the sender becomes the receptor and vice versa. If this main characteristic of the hypermedia system is taken into account in the pedagogical development of new and interactive media materials, classes and courses, future generations will be more able to participate in their own learning processes, thus producing better objective results. In that sense, the hypermedia system is ideal for the construction of culture, since it can be an open system for the exchange and discussion of ideas.
The differential characteristic of hypermedia communication is not only the exchange of functions between the actors of communication as a result of the technological tools, but also the fusion of traditional communications media and digital computing technology (Manovich, 2001: 25), all that allows for a new system of communication that is interactive and multimedia. Any type of expression is potentially able to be used. That is why the multimedia aspect is the one that, with the interactivity, better defines the hypermedia communication system.
A diagram, based on the different educational and communicative experiences analyzed using hypermedia interfaces, has been created for describing the elements and systems that participate in the interactive communications. Presented in Figure 3, it synthesizes in a general, iconographic and schematic way all the elements and systems that interact with and within the hypermedia communication system. It gives us a clear overview of the original aspects of the hypermedia communication system and how it configures a new model of communication and system of interactions.
There is a new dramatic theory for interactive narrative systems developed in several schemas by Mateas (Wardrip-Fruin & Harrigan, 2004: 22), see Figures 4 and 5. The immersion or the mediatory identification is generated in the hypermedia communication through the interaction with the character or through the actions of the reader-author. All the following expressive elements can be interactional; it is possible to do an expressive communicative interaction about them. However, within the hypermedia communication system, it is at the level of the character and the actions that the avatar, the reader-author or player, materializes the expressive interaction, the immersion or the mediatory identification. That is why the expressive elements that appear in the interface are described as elements that serve the interaction and construction of the character or avatar. Shown below, in two schemas, are the differences between the dramatic theory by Aristotle and the updated schema purposed by Mateas.
Mateas (Wardrip-Fruin & Harrigan, 2004: 22-23) gives an explanation of the traditional schema: Aristotle analyzes the works in hierarchical categories, in the center of the schema, corresponding to the different parts of the creation. Those categories are related through formal and material causes. The material cause of something is the material from which is created. The expressive material, from the perspective of communications theory, is compounded by the multimedia expressions and the characteristics of the hypermedia narrative. The formal cause is the abstract plan, the goal or the ideal through which something is directed. The motivational goal, the organized expressions and the narrative structures are created to transmit a plot or theme, which is then associated with certain values. This is also done within interactive media so that it becomes more successful. Kojler (2005: 272) underlines how the most popular Japanese video games have used cinematographic techniques to attract interaction. The video game Donkey Kong from Miyamoto was the first one to introduce a narrative of start/ middle/end in a video game and the narrative structure has been very influential over other games: «from the Electronic Gaming Monthly’s 2001 list from the 100 better video games, 78 had elements of narrative lines and another eight recognized at least clear characters».
In the new model of the interactive drama, Mateas (Wardrip-Fruin & Harrigan, 2004: 22-23) explains «the player has been added to the model as a character who can choose his own actions. This has a consequence».
The player, reader-author, organizes his interaction plan and directs it through the character category.
The player collaborates, or can collaborate, in the construction of the proper narrative forms to communicate a plot or theme, with their corresponding values, coming from both the author and the player of the game. Mateas (Wardrip-Fruin & Harrigan, 2004: 22) continues describing the function of the interactive drama in the game in the following manner «But this ability of taking action is not completely free; it is limited from behind by the material resources…». The interactive multimedia expressions, as well as the spatial and temporal variables, become the constrictions that the game offers, «…and from over it by the formal authorial cause originated at the level of the plot». The author directs the theme of the game or interactive narrative and the main actions of the plot. The English term «affordance», which, when applied to hypermedia can be translated as the availability of interactive multimedia expressions, is also a constriction to the interactions. Nevertheless, this constriction can be made up through the interface design, within a broad variety of hypermedia expressions. When the interaction is attracted by some expressions and developments of the dramatic plot, it would generate a more engaging interaction. The fact that the forms of interaction can be developed through some or all perceptual senses of the player, also means a level of constriction or freedom depending upon the adaptation of the technological tools of the game to the biological senses of the player. This happened for instance with the original Eyetoy, from Sony, most recently with the Wii, from Nintendo, or lately with the Kinect, from Xbox 360.
The original interdisciplinary integration of the previous new models, specially of the previously presented interactive communication and narrative models, lead to the creation of integrated and innovative models for analyzing hypermedia interfaces, which have been proved during the research experiences. In the most recent experience in Peru and Bolivia, a remarkable total of 45 videos were produced in 4 one-week workshops. They were created through the interaction of both first time and initiated audiovisual and multimedia practitioners. A remarkable experience was to observe how the interactive communication system and the new technologies helped the development of media literacy in multiple contexts. Moreover, using an interactive educational model served not only to produce great quantities of videos in a short period of time, thereby letting students learn through completing the whole production process, but also contributed to the quality of the video narratives and techniques: 65% of the videos produced during the workshops had semi-professional quality. Some of the videos were even presented later on in video festivals or even projected on a Bolivian national TV, like the ones titled «Watxia Ritual» or «Paqarina», which are still online on the channel www.youtube.com/multiculturalvideos.
The validity and functionality of the following models of analysis of the hypermedia interface has been proven in several previous scientific researches such as on «Smoke & Mirrors», developed by Sheldon Brown, ex-director of the Center for Research in Computing and the Arts from the University of California San Diego. Moreover, these models of analysis were corrected and they were used as descriptors within the design of an experiment consisting of the observation of the most attractive and interacted expressive, narrative, emotional and value characteristics of the interface for the teenagers. The models were applied also to the analysis of successful video games with some didactic contents, such as Antigrav from Sony, or Sims2 from Electronic Arts. The effective use of the model as an analytical qualitative tool, served to find which immersive combination of expressions and narrative interactive forms served better for the transmission of cultural and educational values to the youth (Mora, 2009).
Moreover, these proven models are presented as conclusive tools, since their application has been very useful in the design of video game and multimedia prototypes. One of them was the Exchanging Cultures EC Game, an educational video game awarded a prize by USC´s Annenberg School of Communication, which was embedded on the MMOG Second Life. This game consisted of an interactive museum, located within SL, where the players were able to learn about different countries and cultures through the exchange of original art, food recipes, architectures, customs and other cultural items. Another recent application of the models was during the interactive media workshops developed in Peru and Bolivia.
The next, Figure 6, summarizes in detail the communicative elements that intervene during the interaction with the interface, where the functional roles are changed from sender to receptor, author-reader or professor-student, and vice versa. These are the interactive dimensions we focused on and observed during the research in order to discover which interactive media expressions improved the immersion or the identification in educational interactive media products. The following Figures 6 and 7 take into account the research of Dr. Isidro Moreno (2003: 110-114) and Mateas (Wardrip-Fruin & Harrigan, 2004: 22) and they are integrated and updated by Dr. Mora and his research experiences from 2003 until 2010, mentioned in the previous epigraphs.
In this schema we can see the elements that compound the hypermedia interface: expressions, narrative forms, emotions and values; these configure the information transferred during the communication human-interface. As Moreno-Muñoz (2000: 57) explains «the human beings as information processors experience a series of levels in the information process». The symbol on the top right represents another similar hypermedia communication system, from other reader-author, who generates another communication system human-interface-human when connected online with another player.
The following, Figure 7 synthesizes the interactive elements and the dramatic narrative processes generated by the player, reader-author, through the interface. It displays the communicative moments when the change of roles between sender and receptor happens, as well as the hypermedia narrative.
In a simplified way, this schema describes the moments when the interface communicates expressions of the different narrative forms. It also illustrates the dramatic narrative structure and how it is organized in the communication between the reader-author and the hypermedia interface. Figure 8 summarizes the generation and process of the hypermedia narrative at the moment of the interaction.
The following model of analysis, developed by the author of the present paper, is the result of the integration of several personal and other previously mentioned research projects. It serves to focus on the hypermedia elements that can be managed during the design and production of immersive and interactive educational projects. The goal of this classification of the hypermedia elements was to create different models of descriptive analysis which could be applied to the analysis of any hypermedia. The model is focused on describing in detail the expressive and narrative characteristics that can be present through the interface. That way it is possible to analyze what types of interactive interfacial expressions can develop different kinds of interaction, as well as with which narrative elements: space, time, characters and/or actions.
Through the exhaustive identification of these interfacial elements, and their dramatic organization, it is possible to develop research projects that observe the different moments developed by the hypermedia communication. These observations allow the identification of the elements of the interface that are more attractive and that motivate the most immersion and identification. Those combinations of expressions, narratives, emotions and values that are more activated and used represent the most effective interface designs. Davenport and the members of the Interactive Cinema Group from the MIT Media Lab (2000: 456-459) emphasize during their research the importance of contents, how the structural form is organized and how the expressive resources are the substantial components for the user’s participation and emotional implication. That is one of the main questions that motivated the creation of a model by the author of this paper. The following model allows observation of what the relationships between the expressions are, each narrative form: action, characters, spaces and times; the types of interaction, and the ethical or unethical values communicated through the hypermedia interface.
The empty cells and the subcategories below some categories serve to describe the qualities of the different characteristics, and are to be filled and/or chosen during the description of the interface analysis. The characteristics of each element that is not applicable can just be deleted, and the ones that are pertinent can be described further. This model has served to implement and increase the multi-sensory immersion and level of entertainment in educational interactive media. The model only includes the descriptions of the terms that might not be found easily in a good dictionary.
In synthesis, the previous innovative models can be applied to implement any educational hypermedia products and media literacy in multiple contexts. It is recommended that the qualitative analyses that the model facilitates be used in combination with quantitative systems allowing the quantification of the interactions with each of the interface elements. One of the most effective is the multi-camera audiovisual recording technique, so the real and virtual actions or the inter-actors are registered in synchronicity. This methodological combination serves to contrast the real interaction facts, developed while using the interfaces, with questionnaires, interviews and group discussions. It also serves to quantify the number of interactions in relationship with the qualitative expressive data described using the analytical model, Table 1.
The analytical process of an in-depth study of hypermedia interfaces is complex and multidisciplinary, especially when taking into account all the different aesthetic, narrative, perceptual, emotional and values variables. The study of broad groups of young adults or of other ages requires a great investment in multiple scientists and experts from different knowledge areas such as: communication, narrative, pedagogy, sociology, cognitive, perceptual and cultural psychology, anthropology, ethics, statistics... With professional teamwork it is possible to broadly apply the model of Table 1 to build a system of producing educational video games and hypermedia systems to different publics. That can be an effective form to create interesting and attractive hypermedia interfaces that could compete with the most popular commercial games on the market. Moreover, more evolved techniques and systematic methods are required for registering the audiovisual data from the physical and psychological behaviors of the interface users. The multidisciplinary research team Neurocinema, at the University of Aalto, Finland, is a good example. This multidisciplinary group is using fMRI for observing how certain movie sequences and interfaces excite some parts of the brain to create more emotional, perceptual and psychological immersions (Wallenius, 2010). This makes it possible to observe information about how to implement effective communications between the different levels of biological and technological media, within the interactions human-human, human-machine or human-machine-human. It will be also necessary to count on programmers, designers, creators, professors and artists for the optimal production of edutainment video games. Public and private investments and collaborations with more formal research groups and academic programs in the area of interactive media communications, narratives and pedagogy are necessary for establishing a solid interdisciplinary field.
Currently, there are very advanced informatics systems, generators of artificial intelligence, virtual reality, of real time rendering which have been developed by the video game industries. The commitment of the video game industry with the development of educational games, and the collaborations between cultural and educational institutions, ministries, universities, and others can be a motivation to exchange knowledge and resources. For instance, the University of California San Diego made some agreements with the video game company Sammy Studios and Sony Entertainment, in addition to others, to sell and/or lend the different informatics resources of video game design for the development of scientific and artistic visualizations. There are many new media companies already willing to establish collaborative bridges for the development of educational applications. The Interactive Media Division, School of Cinematic Arts, University of Southern California, counts on the support of the video game company, Electronic Arts, to develop its Game Lab: a place where professors and students help to innovate and research new video games. For instance, the internationally awarded video game «Darfur is dying» by Susana Ruiz was created there, and served to build on the called «serious games» where the players help the resolution of social conflicts.
Hopefully, more video games and classes based on interactive media systems will be seen, classes that will be more entertaining for the students and a renewal in education for the professors. Institutions and universities such as M.I.T, Massachusetts Institute of Technology, in the Department of Comparative Media, developed educational video games that are good prototypes and examples of the edutainment potential that videogames represent. Games about literature, history, engineering, physics, chemistry and others will fill the classes in the near future, since commercial video games already fill the students´ conversations in the present. Henry Jenkins (2007), head of that Department at that time, lead a virtual multiuser environmental space, The River City Project, to study the relationships and learning transferences developed in augmented realities, mixing the context of the classroom and real life. The adjective commercial or popular will be applicable also to edutainment video games, to those that will offer enough quality. This will be possible thanks to the application of models of analysis in hypermedia interfacing, and the increase of research in the area of interactive multimedia applied to multidirectional educational communication. At the end, what is more attractive in education than learning while playing?
Supports
Funded on 2008-2010 by MAEC/AECID Postdoctoral Research at the UNSAAC and the Museum Convento de Santo Domingo-Qorikancha. Funded on 2006-2008 by MEC/Fulbright Postdoctoral Research Grant; at the School of Cinematic Arts, University of Southern California, Los Angeles California and Laboratory pf Digital Culture (UCM) Cooperation Grants.
Arcila, C. & Said, E. (2011). The Cyber Media in Latin America and Web 2.0. Comunicar, 37, 125-131. (DOI: 10.3916/C37-2011-03-04).
Davenport, G. & Interactive Cinema Group (2000). The Shareable Media Project. Massachusetts: MIT.
Duart, J.M & Osorio, L.A. (2010). Interaction Analysis in Hybrid Learning Environment. Comunicar, 37, 65-72. (DOI: 10.3916/ C37- 2011-02-06).
Gee, J.P. (2003). What Video Games Have to Teach Us About Learning and Literacy. New York: Palgrave MacMillan, 200-212.
González-Faraco, J. & Gramigna, A. (2009). Learning with vi deogames: Ideas for a Renewal of the Theory of Knowledge and Edu cation. Comunicar, 36, 158-164. (DOI: 10.3916/c33-2009-03-007).
Jenkins, H. (2007). Convergence Culture. NY: New York Uni ver si ty Press. (http://muve.gse.harvard.edu/rivercityproject/) (01-03-2009).
Kojler, C. (2005). Power Up: How Japanese Video Games Gave the World an Extra Life. Indianapolis: Brady Games, Pearson Edu cation Inc, 272.
Manovich, L. (2001). The Language of New Media. Cambridge: MIT Press.
Martín-Serrano, M. (1982).Teoría de la comunicación. Madrid: Cuadernos de Comunicación.
Meadows, M. (2003). Pause & Effect: the Art of Interactive Narrative. USA: New Riders: Voices that Matter, 37-39.
Mora-Fernández, J. (2008). Multicultural Videos: An Interactive Online Museum Based On An International ACM Digital Library, Association of Computing Machinery, and the DVD ACM Multi media 2008. WS-Communicability MS2008, 1-8. (http:// dl.acm.org/citation.cfm?id=1462045) (04-01-2012).
Mora-Fernández, J. (2009). Interacciones hipermedia y videojuegos: inmersión comunicativa interfacial y narrativa. Icono 14, 12, 218-241. (www.icono14.net/monografico/interacciones-hipermedia-y-videojuegos) (16-01-2012).
Moreno-Muñoz, A. (2000). Diseño ergonómico de aplicaciones hipermedia. Barcelona: Paidós.
Moreno-Sánchez, I. (2002). Musas y nuevas tecnologías: El relato hipermedia. Barcelona: Paidós.
Wallenius, K. (2010). Video Annotation for Studying the Brain in Naturalistic Settings. Faculty of Electronics, Communications and Automation (pp. 1-4). Finland: Aalto University, School of Science and Technology.
Wardrip-Fruin N. & Harrigan, P. (2004). First Person: New Media as Story, Performance and Game (pp. 2-33). Massachusetts: Institute of Technology.