While not exhaustive, it is necessary to highlight the growing proliferation of reflections from the Hispanic sphere on communication as a field of theoretical study (De-la-Peza, 2013; Fuentes-Navarro, 2017; Moreno-Sardà, Molina, & Simelio-Solà, 2017; Piñuel, 2010; Silva & de-San-Eugenio, 2014; Vassallo & Fuentes-Navarro, 2005; Vidales, 2015). At the same time, numerous efforts have been made to understand the heterogeneous methodological approaches used to conduct research in this prolific area (Castillo & Carretón, 2010; Lozano & Gaitán, 2016; Marí-Sáez & Ceballos-Castro, 2015; Miquel-Segarra, 2018; Ortega-Mohedano, Azurmendi, & Muñoz-Saldana, 2018; Saperas, 2018).
This abundance of contributions is due to the fact that “the interest in meta-research in communication has once again gained strength” (Caffarel-Serra, 2018: 284), and has done so, in effect, backed by two essential factors: first of all, “the search for a system with which to bring order the theoretical findings in the field” (Martín-Algarra, 2008: 153); and, second, the mastery of techniques that enable the attainment of those findings. In this respect, one of the most important collective initiatives in Spain has been the recent “MapCom Project”, to which we will return later by referencing work done by some of its members.
There is, however, a certain dissent around the hierarchy that these two tasks −theoretical reflection and methodological application− hold within academia. In this sense, the most critical voices call attention to “a tendency to continue encouraging a purely instrumental research model, with a certain degradation of theory as an end in itself” (Sierra, 2016: 46). With regard to empirical deployment, and although they are not mutually exclusive, the debate between quantitative and qualitative traditions is no less evocative. There are authors who highlight the rise of a phenomenological perspective, centered on “processes, language and human experience where culture and communication are inexhaustible sources of meanings” (Salinas & Gómez, 2018: 11). From another angle, “the emergence of new research techniques and new tools for the quantitative (statistical) analysis of communication not only constitutes a technical advance, but also substantially affects the development of communication as a scientific discipline and, in particular, has a decisive influence on the development of more sophisticated theories” (Igartua, 2012: 17). In light of the foregoing, the present study emerges with a clear purpose: to X-ray the current state of research in communication through the analysis of papers published in major Spanish and Latin American journals.
State of the art: Identification of previous empirical studies
In this section, a brief chronological overview of the main prior empirical references concerning both the Spanish and international contexts will be made. Beginning with our immediate environment, Caffarel-Serra, Ortega-Mohedano and Gaitán-Moya (2017) focus on the analysis of a representative sample of 288 documents relating to the period 2007-2014: 239 doctoral theses and 49 research projects, reaching the conclusion that 60.71% of the papers are confined to the study of mass media (49.45% traditional versus 11.26% digital).
Along these lines, Goyanes, Rodríguez-Gómez and Rosique-Cedillo (2018) record 3,653 articles published in the 11 leading Spanish journals from 2005 to 2015. Among its many findings, the prevalence of journalism (press and journalistic practices), audiovisual communication (film and television) and studies on audiences and receivers are noteworthy, as they all account for 51% of scientific production. Based on the data provided by some authors and others, we postulate that:
H1: traditional media will have a greater role than digital media.
In one of their latest studies, Martínez-Nicolás, Saperas and Carrasco-Campos (2019) disclose the findings of a content analysis conducted on a large sample of 1,098 articles from the leading communication journals in Spain during the 1990-2014 period, allowing them to trace different research evolution timelines. As the main result of their meticulous work, it is worth noting that almost 80% of the articles constituted empirical research, while 18% were theoretical-conceptual and only 2% methodological. Thus, we contend that:
H2: the works in the sample will exhibit a pronounced empirical approach.
At the international level, Bryant and Miron's study (2004) includes a sample of 1,806 articles published in the journals “Journalism and Mass Communication Quarterly”, “Journal of Communication” and “Journal of Broadcasting and Electronic Media” between 1956 and 2000. After their inquiry, they declare that Framing Theory is the most prominent, followed closely by Agenda Setting and Cultivation Theory. Therefore, it seems coherent to pose that:
H3: Framing Theory will be the most recurring theoretical corpus.
For their part, Potter and Riddle (2007) examine 962 articles from 16 high-impact journals in the period 1993-2005, concluding that 71.4% of the studies reviewed use quantitative methods −where the survey with 32% and the experiment with 29% prevail− and 15.4% qualitative techniques. This logic prompts a new hypothesis, as well as an intimately associated research question:
H4: quantitative methods will have a greater presence than qualitative methods.
RQ1: What kind of samples will the authors of empirical papers use?
With regard to empirical deployment, and although they are not mutually exclusive, the debate between quantitative and qualitative traditions is no less evocative.
Gómez-Rodríguez, Morrell and Gallo-Estrada (2017) focused on the journal “Comunicación y Sociedad”, a leading publication in Mexico. They evaluated a total of 209 papers assigned to the 2004-2016 cycle and identified some recurrent themes, which fall under the labels of: sociocultural environment (43.6%), academic (24.9%), socioeconomic (16.7%) and sociopolitical (14.8%). Therefore, and in relation to the papers pertaining to our study, we ask ourselves:
RQ2: What topics will be addressed most frequently?
Finally, Walter, Cody and Ball-Rokeach (2018) aim to dissect 1,574 articles published in “Journal of Communication” from 1951 to 2016, performing a longitudinal analysis and comparing stages. In the most recent one, from 2010 to 2016, they found that the audience (69.3%) was the main player in the studies as opposed to the message (19.3%), the source and the policies (both with 4.6%). On the other hand, the dominant research paradigm was positivism (87.5%), well above critical and cultural −both of which were evidenced in 4.6% of studies− and rhetorical (3%). Consequently, two new hypotheses emerge:
H5: the main object of study will be the audience.
H6: the dominant paradigm to which studies adhere will be positivist.
Next, the method used in the study will be described, supported by all the preceding empirical initiatives.
Material and method
Objective and sample
The purpose of the study was to outline the state of communication research in the Hispanic sphere through the examination of academic papers. These works, which have constituted the units of analysis, have been grouped ex post to identify, in a statistically significant way, patterns of publication or clusters, also compared, according to their impact factor and geographical origin of the journals where they were included (Spain or Latin America).
For this purpose, a content analysis was performed for being a systematic, objective and quantitative method (Riffe, Lacy, & Fico, 2014; Wimmer & Dominick, 2011), commonly applied to the study of academic texts, as we have seen in the previous section. Initially, one of the key considerations in any content analysis lies in designing a sampling plan (Igartua, 2006) which, in this case, was “multi-stage” (Neuendorf, 2016). Therefore, and in a first phase, the journals selected were those with the greatest impact in 2017 −the latest year for which data are available− in the international database “SJR-Scopus” in the category of communication (www.scimagojr.com). It was determined that journals had to rank in the first two quartiles in order to be rated as high-impact, resulting in a total of seven titles, sorted according to their position in the ranking: “Comunicar” (Q1), “El Profesional de la Información” (Q1), “Communication & Society” (Q2), “Revista Latina de Comunicación Social” (Q2), “Cuadernos.info” (Q2), “Comunicación y Sociedad” (Q2) and “Palabra Clave” (Q2). Likewise, and from 2017 onwards, it was deemed reasonable to go back five years, to 2013, in order to give the sample a certain time perspective.
In short, all those articles were stored −except editorials and reviews− contained in the websites of the seven journals during the period in question, generating a sample of n=1,548 articles. This figure represents 48.77% of the universe of published works (N=3,174) in each and every one of the Spanish and Latin American journals that were indexed in SJR-Scopus in this five-year period, which represented a margin of error of ~1.8% for a 95% confidence interval.
Categories of analysis, coding and reliability
As a guide for the examination of this representative corpus of analysis, and based on other previous studies (Barranquero & Limón, 2017; Caffarel-Serra, Ortega-Mohedano, & Gaitán-Moya, 2017; Gómez-Rodríguez, Morrell, & Gallo-Estrada, 2017; Goyanes, Rodríguez-Gómez, & Rosique-Cedillo, 2018; Martínez-Nicolás & Saperas, 2016; Martínez-Nicolás, Saperas, & Carrasco-Campos, 2019; Saperas & Carrasco-Campos, 2018; Walter, Cody, & Ball-Rokeach, 2018), a codebook was prepared consisting of the following nominal polychotomous variables, along with their reliability indicator:
1) Type of article (αk =0.92): 1=empirical, 2=theoretical/essayistic, or 3=methodological.
2) In the case of empirical work, what method (αk =0.83) is used? (Table 1).
3) In the case of empirical work, what type of sample (αk =0.84) is utilized?: 0=non-empirical work, 1=probability sample, or 2=non-probability sample.
4) Theory (αk =0.70) that provides a conceptual basis for the study (Table 1).
5) Main object of study (αk =0.93): 1=source, 2=message, 3=audience, or 4=policies/structure.
6) Main means of communication or documentary support (αk =0.87) in the article (Table 2).
7) General topic (αk =0.87) of the work (Table 2).
8) Paradigm (αk =0.96) where research is framed (Walter, Cody, & Ball-Rokeach, 2018): 1=positivist: study supported by empirical assumptions and verifiable hypotheses, using quantitative or mixed methods; 2=cultural: qualitative study about the everyday practices that create and sustain culture; 3=critical: study focused on questions of power, political economy, status quo and social structure; or 4=rhetorical: study that conceives communication as the practical art of discourse.
In total, 8 multi-categorical variables in addition to those used to identify the unit of analysis; that is, the number of the article, its publication year and the journal where it appears. Likewise, the SJR impact factor of the journals during the five years examined was collected, assigning to each unit of analysis the average impact factor of the journal in the year in which it appeared. This parameter, which acted as an independent variable, was of great help in profiling the types of articles (or clusters) resulting from the processing of the results in more detail. Finally, the data collection, which ran from September 3 to December 21, 2018, involved a team of two coders. After this process, and in order to check the reliability of their work, a random subsample was selected from ~10% of the cases, which both coders analyzed. The statistical parameter used for the reliability calculation was the Krippendorff alpha (Krippendorff, 2011; 2017), found by using the “Kalpha macro” (Hayes & Krippendorff, 2007) for SPSS (version 24). As can be seen above, the reliability of the eight variables was very satisfactory, while the average rose to M (αk)=0.87 (SD=0.07).
Analysis and results
This section is structured in the following way: an initial section, as a preamble, with the percentage and individualized report of codebook items; and a block where it was used −in duplicate− two-stage cluster analysis, “an exploration tool designed to uncover the natural groupings of a data set” (Rubio-Hurtado & Vilà-Baños, 2017: 118), accompanied by analysis of variance (ANOVA) and contingency tables (χ 2).
Firstly, with regard to the type of article published in the journals from the sample (H2), there is a marked tendency towards the empirical (80.9%) as opposed to the theoretical-essayistic (12.8%) and the methodological (6.3%). Therefore, it is interesting to note both the methodological techniques used in this set of empirical works and the different theories and concepts used in the entirety of the works (Table 1).
If we group the methods into quantitative, qualitative and mixed, we find that the former are present in 46.8% of the papers, the latter in 24.9% and, thirdly, triangulation is present 7.9% of the time (H4). Another important aspect at the methodological level lies in the sample design (RQ1). In this sense, 61.4% of the papers resort to non-probability sampling compared to 19.4% who use statistical criteria for the handling of representative samples −it should be remembered that the remaining 19.1% are non-empirical studies. In relation to the theoretical framework (H3), it is worth pointing out that the proportion of texts that rely on some specific conceptual scaffolding as opposed to those that do not is practically the same: 50.2% as opposed to 49.8%. On an individual basis, Media Literacy (8.3%) and Framing Theory (5.1%) are the most widespread.
With regard to the object of study −in its broadest and most generic sense− (H5), the intermediate link in the process −that is, the message− is the protagonist par excellence (44.6%) of the papers, followed by the audience (21.3%), the source (17.1%) and, lastly, the communication policies and structure (17%). The means or, alternatively, the documentary supports on which the papers focus, as well as their general topics, are distributed as follows (Table 2).
If we aggregate traditional media, we obtain 38.8% of the works compared to 28.2% of digital media (H1). For their part, the themes around which the studies revolve are also the most diverse (RQ2), highlighting the uses and motivations of the receivers (11.3%). To complete the percentage review of the variables in the codebook, we find the paradigm to which the articles are ascribed (H6). In this sense, the one that dominates Spanish and Latin American research in communication is positivistic (56%), at a great distance from cultural (21.1%), critical (14.9%) and, most particularly, rhetorical (8%).
At the multivariate level, a first two-stage cluster analysis was performed which covers both continuous and categorical variables (Rundle-Thiele & al., 2015), in which we included the 6 most relevant items of the codebook; namely: the theory employed in the articles, the method, the object of the study, the communication medium in which they are centered, the topic in question and the paradigm within which the sample studies are circumscribed.
The silhouette measure of cohesion and separation, which alludes to cluster quality and must exceed the minimum level of 0.0 (Norušis, 2012), is 0.2, an acceptable value although regular. On the other hand, the importance of all the predictive items in the configuration of the groups is very high, since four of them reached the maximum value of 1 (topic, medium, object and method), another one was 0.96 (theory) and the definitive one was 0.78 (paradigm).
Subsequently, the 6 clusters resulting from this first extraction presented a size coefficient of 1.54, which denotes some homogeneity between them, as can be seen in Table 3.
Starting from the creation of the 6 groups, these can be compared according to their SJR impact factor. Analysis of variance (ANOVA) shows statistically significant differences between clusters [F6 Clusters x SJR-IF (5; 1,542)=18.66; p<0.001; ɳ 2=0.057]. More specifically, and after Dunnett's T3 post-hoc test, it was deduced that the “3” and “6” clusters are those that show the greatest imbalances [t(528)=−6.903; p<0.001; d=−0.624], labeled as medium size according to “effect size” (Cohen, 1988; Johnson et al., 2008). Also, the 6 clusters were cross-referenced with the journals where the papers are published, recoding them into two groups: Spanish versus Latin American. The following contingency table reflects the distribution of the two according to publications.
In terms of the values shown in Table 4, significant differences are observed in 4 of the 6 clusters [χ 2(5, N=1,548)=87.52; p<.001; v=0.238]. More specifically, and taking into account the corrected standardized residuals, Latin American journals tend to include article types 3 and 5 to a greater extent, while Spanish journals opt for cases 4 and 6.
The second two-stage cluster analysis differs from the previous one in only one of the items used for its execution: the variable “theory”, which was recoded to take into account only those works that employed some kind of conceptual corpus, considering the rest as “missing”. The silhouette measure of cohesion and separation reached 0.3, an even more acceptable value. As far as the importance of the predictive items is concerned, the method becomes the most prominent element (with a value of 1), followed by the object of study (value 0.92), the theory (value 0.52), the topic (value 0.42) and, ultimately, the paradigm and the medium (both with a value of 0.40). The four clusters derived from the analysis have a size coefficient (from the largest to the smallest) of 2.20, which is not problematic (Tkaczynski, 2017). Below is a profile of the four clusters.
These 4 clusters can also be compared according to their impact factor. Thus, this new analysis of variance reveals the existence of differences between groups according to their SJR-IF once again [F4 Clusters x SJR-IF (3; 773)=26.79; p<0.001; ɳ 2=0.094]. Comparatively, and according to Dunnett's T3 post-hoc test, it is clusters 1 and 4 that reveal the greatest imbalances [t(388)=−7.55; p<0.001; d=−0.831], characterized by an elevated size according to “effect size” (Cohen, 1988; Johnson et al., 2008). Finally, these four clusters were cross-referenced with the journals where the articles are published.
According to the values shown in Table 6, significant differences are observed in all clusters [χ 2(3, N=777)=53.33; p<0.001; v=0.262]. On the basis of the corrected standardized residuals, Latin American journals tend to include article types 1 and 3 to a greater extent, while Spanish journals prefer cases 2 and 4.
Discussion and conclusions
In short, the findings derived from this study enable us to assert that, although social networks and ICTs have gained much prominence, conventional media continue to be, in an aggregate manner, the most important in current communication research, which in turn reveals a markedly practical character. Somehow, a certain “lack of reflection on its epistemological dimension, on its conceptual definition” (Vidales, 2015: 12) is noted, appealing in overwhelming proportion to its empirical aspect. In this sense, the most common methodologies are quantitative, although some shortcomings are evident in their more canonical application, especially with regard to the use of representative samples (an unusual practice). From another perspective, the fact that accepted practice lies in the execution of empirical works should in no way imply the precariousness of its theoretical foundation. However, it is astonishing that almost half of the studies do not appeal to a theory or, at least, to some kind of solid conceptual notion on which to base their further research. As for the articles that do refer to some kind of theoretical corpus, Media Literacy stands out, one of the paradigms that obtains greater visibility, for example, in the journal “Comunicar” −although not exclusively−, and Framing Theory, in tune with international trends and other previous studies in the Hispanic sphere (Piñeiro-Naval & Mangana, 2019).
Continuing with this brief summary, and despite the fact that messages are the primary object of study, one theme stands out above the others: the uses and motivations of receivers in their interaction with media artifacts. Bearing in mind that the audience is the second object of study and that the survey is also the second most used method, there is a trend towards greater concern for processes and effects. Nevertheless, until 2017, content analysis studies −or, alternatively, discourse analysis− still hold the message in a prominent place; these works are all encompassed, simultaneously, in the positivist paradigm. The following list summarizes, in abbreviated form, the answers to the hypotheses and research questions formulated:
H1. Traditional Media > Digital Media (accepted).
H2. Empirical Works > Theoretical / Methodological (accepted).
H3. Most recurring corpus: Framing Theory (partially accepted).
H4. Quantitative Methods > Qualitative Methods (accepted).
H5. Main object of study: Audience (rejected).
H6. Positivist Paradigm > Cultural / Critical / Rhetorical (accepted).
PI1. Types of samples in empirical studies? Non probabilistic.
PI2. Most frequent topic? Uses and motivations of receivers.
Another aspect of the work that should be highlighted −perhaps the most novel− revolves around the detection of significant publication patterns, also known as clusters. In short, there are two opposite poles. On the one hand, we find a series of positivistic works focused on the audience and its interaction with ICTs and social media that, on a theoretical level, rely on a Media Literacy trend while, on an empirical level, use surveys to approach the receivers of communication. These papers, usually included in journals of Spanish origin, are the ones with the highest impact factor. On the other side of the scale emerge studies framed in the cultural paradigm that, by means of a qualitative case study, analyze the narrative structure of cinematographic messages −or, alternatively, of serial fiction. These papers have a greater presence in Latin American journals, which implicitly results in a lower impact factor. In short, it seems clear that “research in communication is an object of study that will continue to develop in our country following the trends that are consolidated in a society and a market that are increasingly more communicative” (Caffarel-Serra, Ortega, & Gaitán, 2018: 69). Meta-research will therefore fulfill a fundamental mission in the eclectic field of Communication Sciences: to highlight the shortcomings of the discipline and to warn academics of the risks it will face if not addressed. In light of the results of the study, the most demanding challenges will lie in the two tasks indicated at the beginning of the text: to provide research with a remarkable methodological robustness and, especially, with a rich theoretical repertoire so that its authors can understand and assimilate, with guarantees, the liquid communicative manifestations that flow very quickly from the Information and Knowledge Society.
Other topics include: environment, history, religion, humanism, philosophy, aesthetics, poetry or legislation.
Other media include: photographs, infographs, drawings, graffiti or videos.
Other methods that appear sporadically include: situational analysis, data envelopment analysis or iconographic analysis.
Other theories and concepts specifically identified are: Cultivation Theory, Stakeholders Theory, Priming, Spiral of Silence, Conceptual Metaphor Theory, Grounded Theory, Transparency, Neuromarketing, Gamification, Augmented Reality, e-WoM, Internet of Things, Memes or Think Tanks.
Note that “Comunicar” is also indexed in Education and Cultural Studies, while “El Profesional de la Información” and “Cuadernos.info” appear in Information Sciences, assumed as areas related to Communication.
Information about the MapCom project: www.mapcom.es