Journalistic quality, credibility in the media, and disinformation

Mistrust in the media is often due to people's perception that the media generate disinformation (Masip et al., 2020). The recent economic and business model crisis has contributed to this (Franklin, 2014; Ryfe, 2019), in turn resulting in a reduction in journalistic quality standards (Gómez-Mompart & Palau-Sampio, 2013; Gómez-Mompart et al., 2015). Various studies have highlighted serious deficiencies affecting its social credibility, including a lack of transparency and representativeness; politicisation; hidden agendas associated with commercial interests; superficial coverage of stories; rupture of control mechanisms, and the spreading of inaccurate or biased news (Lacy & Rosenstiel, 2015; Newman & Fletcher, 2017).

The peculiarities of the new hybrid communication context must also be added to the above problems that conventional media has (Chadwick, 2017). In this context, there are actors who have disinformation objectives that are not always easily identified by a public with poor media literacy (García-Ruiz et al., 2014) and who mainly consume news from social networks (Romero-Rodríguez & Aguaded, 2016a; Newman et al., 2021), detached from the informative matrix. This has facilitated the creation and spreading of hoaxes on digital platforms which is particularly significant in the Ibero-American region because of the uncertainty and fear generated by the COVID-19 pandemic (Gutiérrez- Coba et al., 2020). A study on how digital audiences in Argentina, Chile, and Spain assessed online disinformation showed that this group was concerned about biased news and news that had been made up for political or commercial reasons, while they were less concerned about problems associated with a lack of journalistic quality (Rodriguez-Virgili et al., 2021).

These three countries have witnessed the spread of disinformation and the danger it poses to society and security (Salaverría, 2021). During the first months of the pandemic in Argentina, there was growing concern (Nielsen et al., 2020), especially regarding misleading content about COVID-19 and its impact on the country's political and health system (Gamir-Ríos & Tarullo, 2022). In Chile, the group that most trusts social networks and gives the most credibility to false news was found to be the age range of 18 to 24 years (Grassau et al., 2019). Finally, in Spain, the impact of disinformation has been seen in the political (Paniagua-Rojano et al., 2020), scientific (Salaverría et al., 2020), and economic fields (Romero-Rodríguez & Aguaded, 2016b).

Communications students addressing false content

In recent years, a variety of research studies have explored the need for Gen Z to be trained in media literacy, to equip young people with strategies that can develop a critical awareness of the media and, in particular, of digital platforms (Mcdougall, 2019; Silvaira, 2020). Young people use digital platforms more than any other media to access information (Catalina-García et al., 2019), consolidating a trend that was monitored over years (Casero-Ripollés, 2012) and which in turn motivated research into why university students share false content in different countries (Tarullo & Frezzotti, 2021).

Madrid-Morales et al. (2021) attribute a high political commitment and a desire to have fun as the main motivations for exchanging disinformation, while research by Leeder (2019) found that the willingness of university students to share false news through social networks did not depend on the veracity that they attribute to the content. One of the main problems seen is the difficulty in establishing the reliability of sources. Credibility is associated with factors such as the trust generated by the media disseminating the information (Herrero-Diz et al., 2019), or whether it is shared through the individual's interpersonal network, in an essentially playful content consumption, which seeks to maintain and develop contacts with other users (Eger et al., 2020). However, reliability is not a decisive attribute in media consumption, as shown by Pérez-Escoda et al. (2021), who also argue for media literacy from a critical perspective.

Future journalists from Gen Z attract significant attention, both because of their motivation to fight against disinformation and specific training and the implications that their professional performance will have for society. In this vein, certain studies have focused on how Journalism and Communications students assess the current disinformation environment, to analyse their opinions and attitudes towards (Mendiguren et al., 2020; Martín-Herrera & Micaletto-Belda, 2021), their perception of (Catalina-García et al., 2019; Antunes-Sobral & Nina-de-Morais, 2020; Tejedor et al., 2021b) and preparation to identify (Cruz & Morais, 2020) fake news. The results of these works have shown that these students are aware of the disinformation problem, have developed a certain distrust towards the information received, and have sometimes believed certain content that was inaccurate, which leads them to acknowledge that there is a certain difficulty in detecting fake news. This vulnerability seems to have been emphasised with the infodemic crisis caused by the COVID-19 pandemic, a period during which most of these students state having received false news (Martín-Jiménez et al., 2021; Tejedor et al., 2021b). Despite most of this research being limited to national contexts, some comparative analyses have shown variations between countries in the way these students deal with disinformation (Catalina-García et al., 2019) and evaluate social networks as channels that foster its spread (Tejedor et al., 2021a).

The focus of this research is a transnational comparative analysis, and its objective is to explore the skills and abilities of trainee journalists in identifying the content that presents serious problems from the perspective of quality and professional ethics. In this sense, we have developed a two-level comparative study: the first explores first-year (n1) and senior students’ (na) degrees of training, while the second analyses countries – Argentina, Chile, and Spain – with the aim of establishing differences and similarities. The selection of three Spanish-speaking countries facilitates access to the same texts, while geographic diversity allows us to analyse the extent to which proximity to topics ensures greater or lesser ease in detecting problems of quality associated with disinformation. This study aims to answer the following research questions:

• RQ1 What media do Journalism/Communications students use to access information? Do they vary according to educational level or country?

• RQ2 What is their ability to adequately identify journalistic quality? Do these skills improve when they are addressing pseudo-media or inaccurate content?

• RQ3 Do students adequately recognise poor quality?

• RQ4 How well do students believe they can identify pseudo-information (self-competence)? Does it differ from their actual demonstrated ability?

Origin of the media diet

The results of the study reveal that nearly three-quarters of the students who answered the questionnaire use digital media to access information, compared to the other quarter who use one of the three traditional channels (press, radio, and television). However, notable variations were observed, both between countries and between course years within the same country.

The students from Argentina, Chile, and Spain present statistically significant differences (p=0.000499) in their main ways of accessing current event information (Figure 1). While social networks are the preferred means of accessing information for six out of ten Argentinian students, in Spain this proportion drops to 39%, and social networks are used by a quarter of the students in Chile. However, a higher percentage of students (42%) in Chile use digital media as their main channel, which practically doubles the results of Argentina (24%). The sum of digital access translates into a residual consumption of conventional media in Argentina, where barely one in six students opts for television, radio, or the press. This percentage is around a third of the students in the case of Spain and Chile, although with substantial differences in the type of media chosen. While in Chile, the radio plays a prominent role, in Spain and Argentina television is the traditional media with the greatest weight. The printed press is, in any case, a minority in the countries being studied.

In all three countries, there is a growing trend to obtain information through digital channels as the degree course progresses, while the weight of conventional channels decreases. The distribution of responses by the level of training is not statistically homogeneous in all media and social network options: in Spain, radio consumption is irrelevant in senior course years, while press consumption is more popular; a similar trend can be seen in Chile, although the radio remains the most popular; and, in Argentina, the press disappears altogether while the radio increases slightly.

Students’ ability to adequately identify journalistic quality

The percentage of students who correctly recognised the quality of the news – when compared to the values assigned by the expert group – is approximately 40% of the total number of participants who answered the questionnaire (Table 2). The study data allows us to observe much variability in students’ perception of the quality and in the correspondence with news of poor and average quality. The most extreme cases of coincidence between experts and students were with N3 (74%) and N4 (18.7%). In N3, most students identified the poor quality of the news about the former president of Chile, Sebastián Piñera, wearing a Nazi-style bracelet at a formal event. In N4, less than one in five students adequately rated the quality of the news about Pfizer and its vaccine approval. Both pieces of news were published by pseudo-media.

Generally speaking, differences are observed both between countries and course years: the student group from Spain exceeds the average for adequately identifying quality (42.6%), the Chilean group slightly exceeds it (40.9%) and the Argentinian group scored below average (36.9%). The results by course year are consistent with the acquisition of professional skills throughout the degree in Spain and Chile, i.e. the ability of senior students from these countries scored almost seven percent higher. In contrast, in Argentina, there is a drop of two and a half percent in this skill. These results are reflected in the perception of five news items, where first-year students showed greater agreement with the values of the expert group compared to senior students. Conversely, this circumstance was limited to two discrepancies in Spain (N5 and N10) and in Chile (N1 and N8).

RQ2 analyses whether identifying journalistic quality will be more accurate according to the geographical proximity of the students to the information. Of the three pieces published in Argentina (Table 1) – except one, the alleged holidays of the former president of Bolivia, Evo Morales, on an expensive yacht off the coast of Argentina, where he remains isolated – local students showed a higher level of agreement than their peers. In the three items published in Spain, local students showed, on average, greater agreement with the expert group. For their part, in the texts referring to Chilean issues, conclusive results were not obtained: in the news piece about the former president wearing a Nazi-style bracelet, students showed a worse skill-set than the others, despite their results being closer to those of the expert group in the news about the alleged torture centre.

In the four international news pieces, skills correctly identifying quality were significantly varied according to the theme. The Spanish students showed less skill in assessing the items on forest fires in Algeria (N1) and with Pfizer (N4), while they scored closer to the group of experts than those from Argentina and Chile in the texts referring to NASA (N7) and Biden (N12).

The results indicate that there is no significant relationship between the origin of the publication – conventional or pseudo-media – and the students’ perception of the quality of the news. As established by the expert group, the students who answered the questionnaire believed that part of the information published by conventional media met a certain quality level – an average quality – although its recognition (48.1%) was given in a lower percentage than adequate (60%). The evaluation of the pieces extracted from pseudo-media differed more from the benchmark value. Thus, while the expert group established that all the information on these channels had poor quality, the participating students gave an average score to more than half the pseudo-information news pieces (51.2%), which points to difficulties in detecting the poor quality of information regardless of the dissemination channel.

Acknowledgement of errors

With the aim of analysing the ability of students to detect information quality problems in more depth, a specific theme – referring to rights and freedoms, with ethical implications or one that was very controversial – was selected for each country. From Argentina, a news item referring to an abortion that was not penalised (N2) was chosen; from Chile, an alleged case of torture (N11) was used; and from Spain, an accusation that a person of Moroccan origin took over an elderly woman’s house was employed (N9). In addition, two items referring to the pandemic were included: one on the approval of the Pfizer vaccine (N4) and another on Biden's evidence regarding the origin of COVID-19 (N12). The expert group rated the quality of the five pieces as poor and highlighted the specific problems each presented.

The results (Table 3) of the error percentage frequencies correctly identified by the students for the five items indicate difficulties in detecting the problems observed by the expert group, with 25.3% of the total possible errors. The data show that a higher degree of training improves Journalism and Communications students’ competence in detecting information quality problems: the percentage of correct observations ranges from 19.9% for first-year students in Argentina to 34.6% for senior students in Spain. The proportion of identified errors differs significantly across the course year (p=0.02396). If looking at ability by country, students in Spain were able to successfully identify a higher percentage of errors (27.7%), followed by those in Chile (26.7%), while students in Argentina detected a smaller number of the problems highlighted by the expert group (21.4%). Thus, the differences between countries can be considered statistically significant when comparing the results between Argentina and Chile (p=0.000002) and Argentina and Spain (p=<0.00001), although not between Chile and Spain (p=0.3687).

Geographical proximity to an event by students may, in some cases, favour a greater ability to successfully identify quality problems (Figure 2), but not conclusively. Although in the evaluation of the N9 news piece, from Madrid, the Spanish student group showed greater skill than the average in the other countries – including first-year and senior students – they still only correctly recognised 12.6% of the total number of errors. Of the five pieces submitted for evaluation, this was the pseudo-information piece whose quality errors proved most difficult to recognise. In fact, only the absence of contrasting sources (observed by all three groups) and the breach of journalistic ethics (indicated by students from Spain) were significantly identified.

In the same vein, Chilean students showed a greater ability to successfully detect quality problems in the N11 piece – both the first-year (19.5%) and senior (20.4%) students. However, that piece follows the N9 piece, which has the lowest percentage of expert group-student group agreement in Chile (19.9%). In this case, senior students from the three countries correctly recognised that the information did not contrast sources and certain groups also identified inappropriate wording (senior students, Spain) or the presence of speculation to the detriment of the data (Chilean first-year and senior students).

The N2 piece, from Argentina, was the piece that had the highest percentage of respondents correctly identifying its total errors. Despite that, the local student group obtained the lowest rate of correct answers: the country's senior students correctly indicated 49.4% of the inaccuracies, 11% less than the group from Chile, and 27% less than the group from Spain. Most correctly recognised that the news piece did not correctly identify the sources, in addition to not contrasting them (senior students, Chile), was not written appropriately (senior students, Spain), and mixed objective data and opinion (senior students, Argentina and first-year and senior students, Spain).

Finally, the two pieces about COVID-19 offered better results than N9 (9.8% overall average) and N11 (14.7%) in terms of the students' ability to detect information quality problems, being only behind the skills demonstrated in the N2 piece (57%). Senior students from Spain more accurately detected the errors in N12 (31.8% of errors correctly identified) and N4 (20.4%). Students were less successful in the Pfizer news piece (N4), in which the identification of errors ranged between 15% (first-year students, Argentina) and 25.6% (senior students, Spain), especially regarding the mixing of objective data and opinion and the non-verification of sources. In the case of the N12 news piece, the percentage of properly identified errors was higher, ranging from 25.7% (first-year students, Argentina) to 42.1% (senior students, Spain), especially regarding the presence of speculation and the incorrect management of the sources.

Perceived self-competence in identifying disinformation

The students participating in the study express an average to high self-perception of their ability to differentiate information from pseudo-information, with a score of 3.46 out of 5 (Table 4), which represents maximum competence. The chi-squared test confirms the existence of a statistically significant relationship (p=0.000499) between the students' ability to detect disinformation and their country and course year.

The self-competence perceived by the course shows an increase in ability as the course progresses, with an average increase of almost five tenths between first-year students (3.28) and final-year students (3.73) from the three countries. However, the differences between countries and course years are substantial. In Chile, more than a third of senior students believe they are very competent in detecting disinformation, while this figure drops to 25% in Argentina and 15% in Spain. In all three cases, the increase compared to the maximum assessment scored in the first year is relevant. At this level of undergraduate studies, the average abilities (score of 3 and 4 out of 5) offer important differences, with 90% of Chilean students scoring their competence within these ranges, compared to 83% in Spain and 72% in Argentina.

The average self-competence score given to detect disinformation varies among the three countries, ranging from 3.32 (Spain) to 3.67 (Chile). When self-perception of the ability to detect disinformation is compared to the ability to correctly identify quality, the results reveal a major contrast. In the case of Chile, a higher score in being sure of oneself does not have a comparable score in the identification of quality. Among the Argentinian students, there is an average score in certainty, but the actual ability to detect weaknesses in journalistic quality is lower. Additionally, Spanish students have less confidence in their potential but score higher in detecting problems in quality, although the overall results barely exceed the global average by two percent.