A web-based serious game about self-protection for COVID-19 prevention: Development and usability testing


The number of new COVID-19 cases continues to rise rapidly in many countries despite vaccination. The best way to counter the spread of COVID-19 is self-protection. This study documents the development of a web-based serious game (WSG-COVID-19.SP) to promote effective learning strategies for self-protection against COVID-19 and to test the game’s content validity and usability. WSG-COVID-19.SP was developed using situated learning theory and diagnostic feedback mechanism. The game includes six situation storylines with 17 learning objectives. It uses a problem-solving approach to foster practices such as wearing masks, washing hands, and social distancing. Portfolio analysis was used to diagnose learning problems and report on the learning process. An overall summary index—the scale-level content validity index (S-CVI)—was used to evaluate content validity. Usability was tested through a website survey from 71 students from one university to gauge their technological acceptance and the game’s capability to promote future self-protection behaviors. The S-CVI was 0.81. Usability and acceptability were neither related to the users’ college major (whether it is information technology-related) nor to gender. Among the respondents, 84.5% agreed to continue with the self-protection practice as they were motivated by the real-time diagnostic function. The WSG-COVID-19.SP game system has adequate content validity and a high user satisfaction rating.


COVID-19, serious game, usability, interactive scenario, situated-based learning, self-protection

Palabras clave

COVID-19, juegos serios, usabilidad, escenario interactivo, aprendizaje basado en la situación, protección personal


Los casos de COVID-19 siguen aumentando rápidamente en muchos países a pesar de la vacunación. La mejor forma de combatirlo es la protección personal. En este estudio desarrollamos un juego serio de la web (WSG-COVID-19.SP) para promover las estrategias de aprendizaje para protegerse contra el COVID-19. También probamos la validez y usabilidad del sistema. WSG-COVID-19.SP fue desarrollado de acuerdo a la teoría situada de aprendizaje y retroalimentación diagnóstica. Contiene seis historias con 17 objetivos de aprendizaje. Se usa un enfoque de resolución de problemas para promover el uso de mascarillas, lavado de manos y distanciamiento social. Se usó el análisis de portafolio para identificar los problemas y el proceso de aprendizaje. El índice global de validez de contenido de la escala (S-CVI) fue utilizado para evaluar su eficacia. La usabilidad fue probada mediante una encuesta de web de 71 estudiantes de una universidad para evaluar su aceptación tecnológica y la capacidad del juego para promover la protección personal. El S-CVI era 0,81. La usabilidad y aceptabilidad no correspondían con la especialización del usuario (ya sea que esté relacionada con la tecnología de la información) ni con el género. Un 84,5% de los usuarios quería continuar la práctica porque estaban motivados por los resultados diagnósticos. WSG-COVID-19.SP exhibe un contenido válido y una alta satisfacción del usuario.


COVID-19, serious game, usability, interactive scenario, situated-based learning, self-protection

Palabras clave

COVID-19, juegos serios, usabilidad, escenario interactivo, aprendizaje basado en la situación, protección personal


Background and importance

The COVID-19 pandemic has put tremendous pressure and burden on the medical system and has negatively impacted economic development on a global scale (Nicola et al., 2020). Despite vaccinations, the number of newly confirmed cases continues to rise rapidly in many countries (Gardner, 2020; WHO, 2021). SARS-CoV-2 can survive on aerosols and various surfaces and use those as transmission routes (Santarpia et al., 2020). Moreover, an infected person can spread the disease with or without symptoms (Liu et al., 2020). However, as long as people follow the standard cleaning procedures, wear masks, and maintain social distance, the risk of contracting the disease via exposure to various surfaces, the air, or other people is low (Pitol & Julian, 2021); therefore, these three measures are the best ways to protect oneself against COVID-19 (Chamola et al., 2020; WHO, 2021). A study has found that the combined effect of adhering to all three measures is more powerful than using one measure alone (Álvarez-Pomar & Rojas-Galeano, 2021). The challenge for health educators, then, is to promote the implementation of these correct protection measures to prevent COVID-19. In 2020, distance education became ubiquitous as a result of the pandemic (Kim, 2020). The increasing popularity of e-learning and internet-based applications has greatly expanded the possibilities for learning, and the use of these innovative media has been trending upward (Chamola et al., 2020). Multimedia, including text, image, video, and audio (TIVA), has been widely used for health education on COVID-19 prevention websites (CDC, 2020; WHO, 2021) because TIVA-based materials are quick to produce, easy to use, and inexpensive. However, there are still many disadvantages in using TIVA for health education (Nayef, 2015). For example, it only delivers one-way information transmission, provides less diverse and uninteresting content, and it is unlikely to motivate the user to actively seek information (Garris et al., 2002). Additionally, it is difficult to provide scenario simulations in such a learning environment, which yields a lower learning retention rate and offers fewer opportunities for putting learning into practice (Brown et al., 1989). The difficulty in conducting assessment and diagnosis and the inability to correct the learning process are also significant barriers (Blackburn & Hakel, 2006). Thus, it is a challenge to design self-learning material to compensate for and overcome the limitations of TIVA and to improve the outcome of learning about self-protection against the pandemic. The use of a serious game, involving both entertainment and education purposes, is a useful model to address the aforementioned issues and has been applied to COVID-19 prevention through the quiz format to assess performance (Gaspar et al., 2020; Suppan et al., 2020).

Study aims

The current study developed an interactive scenario-based web serious game (hereafter, WSG-COVID-19.SP). The purpose of the game is for the users to learn about the self-protection measures against COVID-19. WSG-COVID-19.SP adopts four learning mechanisms: game scenarios for increasing learning motivation, simulated practice to build self-protection skills, diagnostic feedback to increase understanding, and portfolio analysis to identify misconceptions and problematic behaviors in COVID-19 prevention. WSG-COVID-19.SP is expected to motivate users to implement self-protection measures. It is also expected to be useful for preventing diseases with similar transmission models. Therefore, the content validity of the game was evaluated by involving healthcare workers. Usability testing was done to determine the degree to which the game enabled the users to learn about the self-protection measures and use them continuously. The research questions are as follows:

  • Are the contents of WSG-COVID-19.SP valid and satisfactory?

  • Do users' college major (information technology-related or non-related) and gender affect their perceived system usability?

  • Can WSG-COVID-19.SP identify user misconceptions and problematic behaviors in COVID-19 prevention by analyzing the learning records?

Literature review

Game-based learning

Motivation is a critical factor for effective learning. The element of uncertainty in games can increase and maintain the participation of players (Kim, 2020) and arouse their psychological need for quickly overcoming the challenges and reaching the goals (Freitas, 2018). As an instructional method, games are a valuable tool for enhancing learning outcomes. When learners play an enjoyable game, they often assimilate the game’s viewpoints. Therefore, the effectiveness of digital game-based learning (GBL) is a proven strategy to increase learning interest and motivation in a myriad of subject areas including mathematics, business, computer science, and language (Park et al., 2019). Using GBL as the foundation, serious games (SG) possess the entertainment characteristics of gameplay but are used for learning or training purposes (Wattanasoontorn et al., 2013). SG has also been applied to many domains for knowledge building, such as conducting efficient training in the field of health, e.g., using SGs with elderly people for disease prevention and rehabilitation (Wiemeyer & Kliem, 2012).

As a response to COVID-19, Suppan et al. (2020) developed "Escape COVID-19," a serious game that promotes safe practices for healthcare workers. The game uses single choice, multiple choices, and drag-and-drop test items, along with the corresponding feedback after answering. Similarly, Gaspar et al. (2020) developed a mobile serious game that uses a quiz format to assess user performance and cartoon-style cards to enhance motivation. They used Google Analytics to analyze the statistics of the game use. Their purpose was to educate and reach out to young players with scientific information about personal care relating to COVID-19 prevention. However, such an interactive scenario and practice experience may be insufficient if a game interacts with the players through choice questions only.

Situated learning theory and scenario-based learning

Multimedia materials (e.g., video, audio, and animation) have been widely adopted in teaching and training. They facilitate self-learning effectively because learners tend to find the content attractive (Chang et al., 2010). However, to offer learners an authentic learning experience, multimedia alone is insufficient (Kinshuk et al., 2016). Situated learning theory is a constructivist cognition theory that emphasizes learners’ need for an authentic learning context (Brown et al., 1989). The theory proposes that learners should be placed directly in an authentic scenario, a manually set up simulation setting, or a digitally produced virtual simulation scenario so that they can learn the targeted skills by continuously interacting with people in the different scenarios. Scenario-based learning (SBL) follows the principles of situated learning theory (Naidu et al., 2007) and it uses interactive scenarios to enable learners to demonstrate their decision-making process. With technology and a sound framework, SBL can motivate students to take the initiative in their learning and provide them with an instructional environment to safely construct and solve real-world problems (Vlachopoulos et al., 2017). For example, Richardson et al. (2017) applied the SBL approach to assist nursing and midwifery students in acquiring the appropriate attitudes and knowledge related to sustainability and climate change. Similarly, Torkshavand et al. (2020) adopted simulation-based learning to facilitate students’ knowledge and skills for elderly patient care.

Learning portfolio analysis

A learning portfolio is a collection of a learner’s work, organized in a way that supports the instructional process and evaluation (Alzouebi, 2020). It can enhance self-regulation and goal-oriented motivation (Šliogeriene, 2016) and effectively document and manage the learning processes and analyze them in real time. For instructors, learning portfolios can help identify the blind spots in their instruction, while for learners, comprehensive and detailed documentation of each step in the learning process provides an opportunity for reflection (Kim, 2020). Learning portfolios can be used, for example, to assess software operating skills (Su & Lin, 2015), investigate the differences in learner behavior and preferences by employing multi-platform learning analytics (Ruipérez-Valiente et al., 2020), or personalize learning with adaptive scaffolding (Su, 2020). In this study, the users’ learning and operational portfolios were analyzed to identify their learning problems to facilitate self-learning.

Design and methods

Theoretical basis

Rooted in situated learning theory and employing interactive scenario mechanisms, WSG-COVID-19.SP (Figure 1) is designed according to the following propositions. 1) An interactive scenario is used to increase the users’ motivation to revisit the TIVA-based information and be repeatedly tested using the simulated assessment to help them understand their weaknesses (Bardach et al., 2020). 2) Simulated practices (Torkshavand et al., 2020) can improve users’ knowledge about COVID-19 prevention and their ability to apply that knowledge in their daily lives. 3) Diagnostic feedback (Su, 2020) can enhance users’ understanding of epidemic prevention information and the necessary self-protection measures. WSG-COVID-19.SP has the functionality of storing all operational and assessment records, allowing the users to learn from the system-generated evaluations.


Concepts and assessment indicators of self-protection against COVID-19

The study designed the learning activities and assessment indicators (AI) (Table 1) in WSG-COVID-19.SP based on the information about the necessary self-protection measures against COVID-19 provided by WHO (2020) and CDC (2020). One AI example is 1.1.a, which deals with the concept of “face masks,” including “how to choose” the appropriate mask. The corresponding learning indicator is “surgical mask.” All AIs were evaluated and confirmed by medical professionals in fields such as infectious disease control, public health, family medicine, and nursing.


Definitions of scenario tasks in WSG-COVID-19.SP

There are four types of scenario tasks in the game, which are designed to simulate the knowledge and behaviors related to infection prevention that should be put into practice in daily life. The tasks are defined as follows. 1) Required task: the primary task related to epidemic prevention. 2) Sudden task: a sudden event that may occur in daily life. 3) Optional task: a task that is related to the required task but is optional. 4) Pass task: the task that determines the outcome of the game. Each task is associated with the prevention-related concepts defined in Table 1. The scenarios in WSG-COVID-19.SP were constructed as shown in Figure 2.I. The required task is the core of each game scenario and is supported by the sudden and optional tasks. The performance on these three tasks determines the result of the pass task and the user’s success in completing the game level.

Design and components of WSG-COVID-19.SP

The design of WSG-COVID-19.SP consists of six steps (Figure 2.II), each with a specific objective. Steps 1 and 2 create the core game scenario based on the scenario tasks (a required task, a sudden task, and an optional task as defined in Section 3.3). The pass task uses the “forehead temperature” as the primary condition for passing the level. The tasks in the scenarios are all connected to the essential epidemic prevention concepts to assess whether users can complete them by properly applying their epidemic prevention knowledge. The pass task evaluates users’ understanding of epidemic prevention, and the score is represented by the forehead temperature bar on the screen (see, for example, Figure 3.I.c). During the game, all responses to the scenario tasks are documented by the system to enable the analysis of conceptual understanding and behavior.

Subsequently, Step 3 takes the pass tasks from Steps 1 and 2 and determines the outcome of the game. Then, Step 4 conducts an automatic analysis of the system’s record of user responses through the diagnostic feedback mechanism. The system then produces a report on the accuracy of the responses in the epidemic prevention scenarios to help users understand their learning status. In Step 5, after the game scenarios are completed, the users are provided with a multiple-choice test to assess their epidemic prevention knowledge. In essence, their conceptual understanding is evaluated again using a different format.


Furthermore, a report is also provided for this second test to help the users reflect on their learning. They can examine their misunderstandings and reflect on their COVID-19 prevention knowledge based on the two personalized diagnostic results (the game report and the test report). They may choose to go through WSG-COVID-19.SP again to correct their previous conceptual and behavioral mistakes. In short, a scenario-based game with a variety of tasks can motivate users to self-learn and self-assess repeatedly to correct any misconceptions and problematic behaviors by re-playing the game. This cyclical self-learning model of “playing, assessment, diagnostic feedback, understanding the problem, practice, and correction” can help users implement COVID-19 prevention and self-protection measures more effectively.

Usability testing and result analysis


Content validity evaluation by healthcare workers

Six experts were involved in evaluating the validity of the knowledge and content related to COVID-19 prevention. They represented several health-related fields: a family physician, an infectious disease physician, two public health professionals, and two nursing professors.

Usability testing by field users

As a web-based game system, WSG-COVID-19.SP is published on the university website by the research team. The study participants completed the usability questionnaire on their mobile devices within a month of using the game system. Participation was voluntary, and participants could withdraw from the study at any time. At the beginning of the research activity, a general introduction about the purpose of the study was provided to all the participants, and their consent was obtained. In total, 71 users (college students) completed the game system survey. The number of male and female participants was comparable.

Usability testing tool

The users completed a self-administered questionnaire on a mobile device to provide feedback on their experience with the system. The questionnaire was designed based on the Technology Acceptance Model (TAM) (Davis, 1989; Venkatesh & Bala, 2008). It investigates the users’ behavioral intention to use WSG-COVID-19.SP and the feasibility of promoting this game. The study also examined two external variables—users’ perceived comparison and diagnosis (Liaw, 2008)—to determine the effect of the scenarios. Additionally, we assessed the cognitive load that the system placed on the users (Sweller et al., 1998).

Table 2 defines each item in the questionnaire. The questionnaire contains eight measurement scales and uses a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). The internal consistency of the questionnaire was reliable (Cronbach’s alpha=0.903).


Learning process in WSG-COVID-19.SP

TIVA-based information materials

After reading the TIVA-based materials related to COVID-19 on the research website, users enter WSG-COVID-19.SP to further their self-learning and self-assessment through interactive scenario-based learning, making up for any deficiencies in the TIVA-based materials. The COVID-19 prevention knowledge on the current TIVA website includes four domains:

  • Knowledge of COVID-19: global pandemic trends and the mechanism of the disease.

  • Taiwan’s policies and regulations related to COVID-19 prevention.

  • Guidelines and principles for basic self-protection measures.

  • Environmental risk factors related to COVID-19 and the appropriate disinfectants.

Entering WSG-COVID-19.SP

In WSG-COVID-19.SP, each screen and storyboard displays the interactive tools and scenario-based learning prompts as shown in Figures 3 to 5. As users enter the game, they are given instructions about the learning objectives (Figure 3.I.a). Furthermore, a menu can be accessed at any time (Figure 3.I.b–c), as well as the required task for the day (Figure 3.I.c). There are two days’ worth of scenario tasks (see Figure 3.II for Day 1 and Figure 4.I for Day 2; also see Steps 1 and 2 in Figure 2.II). Note that the scenarios require the users to practice social distancing in places such as theaters, department stores, and clinics.

Game outcome indicator (the pass task)

In the game, forehead temperature is used as a game outcome indicator. It reflects the accuracy of the user’s responses and determines the user’s success or failure. In other words, the pass task is to keep the forehead temperature under 37.5 °C. If the forehead temperature on the screen is <37.5 °C, then the level is completed successfully, and vice versa. Whether the forehead temperature is displayed is up to the user. The game’s open design allows users to exercise their judgment and critical thinking. The game does not display the forehead temperature by default at the beginning (Figure 3.I.e). However, every time users enter an indoor space, they are asked if they would like to measure the forehead temperature.

For infection prevention and self-protection, the assessment items include mask-wearing, the right time to wear a mask, the correct handwashing technique, and the right time to wash hands. Take mask-wearing for example. The first thing users need to do upon entering the game is to visit the pharmacy and select a mask before tackling any of the tasks (Figure 3.I.h). If a user forgets to buy a mask, he or she will be at risk of infection, and the forehead temperature bar will show a fever as the user works through the tasks.

Scenarios and learning tasks in WSG-COVID-19.SP

Five scenarios (a hospital, public park, classroom, bookstore, and restaurant) and one additional sudden task were designed for Day 1 (Figure 3.II) and Day 2 (Figure 4). Supplies such as masks and alcohol-based sanitizers are shown on the screen. Users use their mobile devices to interact with the virtual people or events in the scenarios to complete the tasks.

In the process, they need to apply their TIVA-based knowledge through critical thinking and problem-solving skills to deal with the problems and situations they encounter. They can protect themselves by carrying out appropriate actions to prevent infection. To attract and maintain user attention, the game scenarios are designed such that both the difficulty and the fun increase as the challenge level goes up. Users must perform more actions in the task to achieve the learning objectives. The self-protection measures are repeatedly shown in various scenarios to ensure deeper learning. If the users do not demonstrate the correct self-protection behaviors in the game, they are at risk of infection.

DAY 1 scenario

For Day 1 of the game (Figure 3.II), users complete up to two actions for each task to achieve the learning objectives for self-protection.

  • In the hospital scenario, the learning objectives include “taking the forehead temperature” and “wearing a mask” when visiting patients in the hospital.

  • In the public park scenario, the learning objectives are “how” and “when” to use alcohol-based sanitizer to disinfect the hands before touching the eyes and nose.

  • In the classroom scenario, the learning objectives are “taking the forehead temperature” and “how to wash your hands" before eating.


Day 2 scenario

The game scenarios for Day 2 (Figure 4) are more challenging than those for Day 1. Users have to perform more actions for each task to achieve the learning objectives.

  • In the bookstore scenario, the two learning objectives are “using tissue paper” and “disinfecting the hands with alcohol” after sneezing.

  • In the restaurant scenario, the tasks are “taking the forehead temperature” and “wearing a mask” when entering the restaurant, as well as washing the hands correctly (Figure 4.II) before eating.

  • Furthermore, a sudden task is added to promote deeper learning. The learning objective of the sudden task is decision-making regarding “wearing a mask” and “choosing a safer place”.

Personalized diagnostic report

After users complete the interactive simulation scenarios, the diagnostic feedback mechanism in WSG-COVID-19.SP automatically analyzes their learning portfolios to provide a personalized diagnostic report, which enables them to understand their misconceptions and incorrect practices. Based on the diagnostic results, the system then presents one of three possible outcomes: safe (low risk), moderate risk, and high risk (Figure 5.a).


The diagnostic report of performance (Figure 5.b) provides individualized results based on the items listed in Table 1 to help users understand their errors. Additionally, the game system uses choice questions to test their basic infection prevention knowledge (Figure 5.c), followed by a report of the test results (Figure 5.d). Consequently, users are motivated to further pursue the relevant knowledge and re-test themselves. As such, this model can promote users’ COVID-19 prevention knowledge and self-protection measures.



Content validity

The experts defined and evaluated the content and scenarios of the proposed WSG-COVID-19.SP scheme from three perspectives: 1) the specificity of the content, 2) practicality of the method, and 3) readability of the wording. They evaluated each item in the game using a 4-point Likert scale ranging from 1 (not appropriate) to 4 (appropriate). The mean score of each item ranged from 2.8 to 3.4. The content validity index (S-CVI/Ave) for the overall scale was .81, indicating adequate content validity.


This section presents the results of the website questionnaire on the usability of the system. We examined the item mean differences between the users in information technology (IT)-related majors and those in non-related ones, and between males and females (see Table 3).

The usability of WSG-COVID-19.SP was rated by all 71 users. The results show that the users were interested in the game and perceived enjoyment while playing (Mean±SD: 4.16±0.77). Furthermore, they rated the system as providing a better situated experience, and better conceptual learning and assessment (4.28±0.74) compared with TIVA-only materials. Because WSG-COVID-19.SP can automatically generate a diagnostic report at the end of a learning session, the users rated the diagnostic feedback mechanism (4.28±0.74) as highly useful.

WSG-COVID-19.SP was designed with the following functions in mind: game-based enjoyment, effective assessment and learning, and a meaningful diagnostic report. As expected, the users perceived the game system as useful (4.33±0.67), highly satisfactory (4.35±0.65), and easy to use (4.09±0.79). They also indicated a high intention to use it in the future (4.28±0.68). Moreover, except for cognitive load, all indicators in the usability test were rated at least 4.0 out of 5, and the scores were not affected by user background. While cognitive load was rated as moderate (2.92±1.10), it is worth noting that the users in an IT-related major experienced a significantly lower cognitive load (2.26±1.13) than those in a non-related major (3.18±1.03). Thus, “user major” is an independent variable that affects cognitive load. However, the same cannot be said of “gender” because while females appeared to have a lower cognitive load (2.7±0.89) than males (3.08±1.23), the difference was not statistically significant.


Portfolio analysis

To investigate user misconceptions and learning problems, the error rates of the assessment indicators (AI) (Table 1) in the learning records were analyzed. Based on the change of error rates between the first and the second system use, three types of conceptual change can be defined: 1) Less problematic concepts, with both error rates being ≤20% (AI-3.1.b, AI-3.1.c), 2) concepts that need to be strengthened, with both error rates being ≥ 30% (AI-2.1, AI-2.2.b, AI-2.3, and AI-3.1.a), and 3) improved concepts, with the first error rate being ≥30% and the second being ≤20% (AI-1.1, AI-1.2, AI-2.2.a, AI-2.3.a, and AI-2.3.b). It was found that the concepts to be strengthened (type 2) were mainly related to “handwashing” and the concepts with significant improvement (type 3) were related to “How to wash your hands?”.


In the current study, WSG-COVID-19.SP can be referred to as a serious game because it was designed for a primary purpose other than pure entertainment. Its content focuses on acquiring and assessing the knowledge and skills needed for self-protection during the COVID-19 pandemic (Gentry et al., 2019).

In the usability test, the 71 users rated WSG-COVID-19.SP with a score of at least 4.0 out of 5 on almost all indicators. Except for the cognitive load indicator, these scores were not affected by the users’ college major, a variable that we use as a rough indicator of their IT competency. The scores were also not affected by their gender. Overall, the users indicated a high intention to use it in the future. They also thought that the current system design provides game-based enjoyment, effective assessment and learning process, and meaningful diagnostic reports for self-reflection and self-correction.

The high level of usability can be attributed to the following factors. 1) WSG-COVID-19.SP presents reliable information obtained from the TIVA-based COVID-19 materials published by public agencies on reputable health information websites (CDC, 2020; WHO, 2021). 2) It was developed based on appropriate theories including situated learning theory (Brown et al., 1989; Naidu et al., 2007), problem-based learning (Vlachopoulos, & Makri, 2017), and digital game-based learning (Park et al., 2019) to apply learning portfolio analysis techniques (Alzouebi, 2020; Su & Lin, 2015) and produce diagnostic feedback (Su, 2020). The adopted theories and design elements enhance self-regulation and goal-oriented motivation (Šliogerienė, 2016) and improve users’ understanding of epidemic prevention information and the necessary self-protection measures.

In this study, cognitive load was the only dependent variable affected by users’ IT competency. When using the system, the users in an IT-related major experienced a significantly lower cognitive load than those who were not. It is known that IT skills may affect users’ perceptions and acceptance of a digital system (Lopez-Sintas et al., 2020). IT competency can be improved through repeated practice. The gender of the users had no significant effect on their cognitive load. Regarding IT competency and gender differences, our findings are similar to those from a previous meta-analysis study (Siddiq & Scherer, 2019). Specifically, the gender gap may not be as large as previously expected in the domain of information and communication technology.

An interesting finding related to how the WSG-COVID-19.SP system uses portfolio analysis to identify common misconceptions is that the concept of “When to wash hands?” had the highest error rate at 51% (AI-2.1, see Table 1), followed by “hand-rubbing procedures” (AI-2.2, a 26% error rate) and “handwashing procedures” (AI-2.3, 33%). Even though the standard procedure of handwashing has very rigorous requirements and must be followed strictly to be scored as correct, surprisingly, the most common error was about the right time to wash the hands, with an error rate as high as 51%. This indicates that most people tend to be complacent about this particular epidemic prevention measure, forming a weak link in the chain of epidemic prevention efforts. For example, for the concepts of “Choosing a mask” (AI-1.1, 17%), “When to wear a mask?” (AI-1.2, 29%), and “Monitor your health daily (taking the forehead temperature)” (AI-3.1.a, 67%), users may know the information but still neglect to perform the correct actions in practice. Thus, through the use of simulated scenarios, users may become more vigilant about epidemic prevention in their daily lives. These portfolio analysis findings identified the concepts that needed a better promotion campaign, a functionality that cannot be provided by TIVA-based materials or existing serious games for COVID-19 education as they have yet to take into account these error-prone concepts and behaviors (Suppan et al., 2020; Gaspar et al., 2020).

The aforementioned results effectively answer the research questions listed in the introduction section. Therefore, it can be concluded that WSG-COVID-19.SP demonstrates a valid serious game design with adequate content validity (S-CVI/Ave=.81) (Polit & Beck, 2006). Additionally, it is easy to use and usable for learning the concepts and skills needed for self-protection against COVID-19. Consequently, and importantly, users will likely continue to use this serious game system in the future.


This research is still in the developing stage for the serious game WSG-COVID-19.SP for learning self-protection skills against COVID-19. The game’s content validity and usability are continuously being tested. Note that the results presented in this paper do not confirm the effectiveness of this game in improving actual real-world self-protection or successful application to other populations.

The sample used for the usability test was a small number of college students. If the game were to be promoted in hospitals and used as an assistive device for healthcare professionals in health education, potential hospital users including the general public and clinic outpatients should be included in future research samples.


This study provides evidence of the usability and user perceptions of the WSG-COVID-19.SP game system, an interactive scenario-based web application designed to empower people in preventing COVID-19 through situated practices. Usability and acceptability were evaluated and reported as adequate by the users. The participants perceived WSG-COVID-19.SP as useful for promoting the self-protection measures against COVID-19 due to the game scenario design and the diagnostic feedback feature. In addition, the portfolio analysis can indeed identify the problematic areas in users’ understanding and practices of COVID-19 prevention measures. Based on these findings, we plan to design an experimental study in the future to examine the effects of the game on self-protection behaviors in real life. This serious game can be applied to prevent other infectious diseases with similar transmission models and to test the effects of their corresponding prevention campaigns.

Author Contribution

Idea, J.M.S., C.J.W.; Literature review, J.M.S., C.J.W.; Methodology, J.M.S., C.J.W., Y.C.Y., T.N.W.; Data analysis, J.M.S., M.J.L.; Results, J.M.S., C.J.W., Y.C.Y., T.N.W.; Discussion and conclusions, J.M.S., C.J.W., Y.C.Y., T.N.W. M.J.L.; Writing (original draft), J.M.S.; Final revisions, C.J.W.; Project design and sponsorshop, C.J.W. (1)