This study was built upon our formative evaluation work that identified common rhetorical strategies and COVID-19 vaccine misinformation narratives and used formative surveys to explore their prevalence and validate select survey items that were used in this study [20,51,52]. The narratives were identified by analyzing 6 months of content from 10 online channels of antivaccine or COVID-19 denialist propaganda. These took the form of Twitter accounts, amateur videos, documentaries, Facebook groups, blogs, and Instagram pages. From these media sources, we created a list of 22 key narrative tropes and 16 rhetorical strategies, which represented the discursive foundation of the antivaccine and COVID-19 denialist media data collected, and created a codebook [53]. Narratives ranged from general claims that the COVID-19 vaccine could cause physical injury to the theory it was a bioweapon promoted by intelligence agencies for shadowy and perhaps even supernatural purposes. Some rhetorics framed their arguments along the lines of bodily autonomy by co-opting the language of women’s reproductive rights, while others relied on audio-visual cues, such as nauseating colors and low-frequency sounds, to cue unease in their audience.

Based on the identification of the antivax narratives, we selected a prominent metanarrative related to vaccine injury that was used to develop 3 different inoculation messages: (1) a fact-based video, focused on countering false statistics about the science and safety of vaccines; (2) a narrative and rhetoric-focused video, which “prebunked” (ie, practice of countering potential misinformation by warning people against it before it is disseminated) common antivaccine misinformation strategies; and (3) a hybrid video that tested a combination of factual rebuttal with narrative/rhetorical prebunking. These 3 approaches were selected in order to deepen understanding of the relative efficacy of fact-based and narrative/rhetoric-based persuasion. As discussed before, the relative efficacy of each approach has been addressed across a variety of fields, from extremist deradicalization to public health.

We designed a series of 30-second inoculation videos and conducted a quasi-experimental study to test the use of attitudinal inoculation in a population of individuals who were unvaccinated. We developed 3 inoculation videos. Each 30-second video contained a “microdose,” a weakened example of manipulation, which has been shown not to cause harm in a controlled research setting. The microdoses, while weakened, constitute an “active threat” that let people generate cognitive “antibodies” [40,54,55]. The differences between the 3 inoculation videos are shown in Table 1.

Based upon our prior research on the relationship between knowledge, attitude, and behavior, we posited 3 hypotheses:

We conducted a quasi-experimental study with a pre-post intervention questionnaire and control group using an organic sampling survey method between June 3 and 5, 2021 [56]. Using this method, 4 separate surveys were conducted for each of the video exposure interventions, and as such, randomization was not possible. We conducted our surveys through the Pollfish (an online survey company) survey platform. Participants were eligible to participate if they indicated they had not received a COVID-19 vaccine, were over the age of 18 years, and lived in the United States. In total, 500 US adults were recruited into each study arm by Pollfish via mobile technology. Respondents were recruited into 1 of 4 study arms, 1 for each type of inoculation message, plus a control group that received a video unrelated to inoculation or vaccines using the design outlined Figure 1.

All participants first answered questions about their demographics, social media and information consumption, exposure to and trust of information about COVID-19 vaccines, and perceptions of vaccine harm. Then, participants in the 3 treatment groups were “inoculated” by showing them a 30-second scripted video that highlights narrative or rhetorical tactics used in vaccine misinformation (intervention group 1, “Narrative,” n=500), contains factual rebuttal of vaccine misinformation (intervention group 2, “Fact,” n=500), or a hybrid of both (intervention group 3, “Hybrid,” n=500). The control group (n=500) watched a neutral video that described how to make a paper airplane. Participants were then asked a series of questions on their perceptions of the video.

After watching the inoculation video, participants were shown a video stimulus that utilizes the manipulation techniques participants were alerted to in the inoculation video. The same stimulus video was displayed to participants in the control group and all treatment groups. Participants were then asked the same series of questions on their perceptions of the video. The same questions were asked following the intervention/control video and the stimulus video in order to avoid alerting the respondent as to the type of video being assessed.

The study protocol was approved by the American University Institutional Review Board (IRB-2022-295).

We had 3 hypotheses of interest. Each hypothesis corresponded to a different dependent variable below. The first hypothesis was related to the ability to identify misinformation, the second was related to willingness to share information, and the third was related to willingness to get vaccinated. The measures described below were designed to correspond to these 3 research questions, though the survey also included other items related to emotional reactance to the content.

The analyses included 3 steps: first, to conduct descriptive statistics of baseline sample characteristics; second, to analyze multi-item scale development; and third, to test the effect of the 3 interventions in comparison to controls on the endpoints collected after the second video. All final models were adjusted for demographics (age, gender, race, education, and income); baseline value, if available; and mean scores calculated about general attitudes toward vaccines.

This analysis included 1991 subjects. Although 500 (25.1%) subjects were enrolled in each intervention group, not all subjects completed the survey questionnaire, leaving an analysis population of 495 (24.9%) participants in the control group and 480 (24.1%) in the narrative-rhetorical, 489 (24.6%) in the factual, and 489 (24.6%) in the hybrid video intervention groups. Overall, the study population had a mean age of 40.7 (SD 11.8) years, and 968 (50%) were female, 1439 (74%) were non-Hispanic White, 1173 (60%) had a bachelor’s degree or higher education, and 773 (40%) reported an income of US $100,000 or more. A summary of baseline characteristics for the entire sample and by intervention group is given in Table 2.

PCA of the 7 items measuring prior attitudes toward vaccines retained 2 factors that accounted for 65% of the variance in the data. All items had a high factor loading on 1 of the 2 factors, with loading in the range of 0.71-0.83, while also having small factor loadings of less than 0.2 on the other factor (Table 3). Items loading most heavily on the first component asked the level of agreement to the statements “Most vaccines do not cause immediate injuries or side effects,” “Most vaccines do not lead to long-term side effects,” and “Vaccination can protect me from getting a disease,” while the remaining items that asked “Vaccines cause more harm than benefit,” “Taking a vaccine is likely to give me a disease,” “Vaccines cause autism,” and “Vaccines are designed as a form of government control” loaded highly onto the second component. Cronbach α for the 3 items loading on the first component was .70, while the 4 items loading on the second component had a Cronbach α of .83, indicating good internal consistency within each set of items. Because these 7 items formed 2 distinct constructs with good reliability, we created 2 scores for general attitudes toward vaccines by taking a subject’s mean response to the questions that loaded highly onto each factor.

For the 3 questions measuring the Recognize outcome, PCA showed the items to be unidimensional, with a single factor accounting for 72% of the variance. Factor loadings were all greater than 0.84, and Cronbach α was .81. PCA of the 5 items measuring the Share outcome also retained a single factor that accounted for 70% of the variance in the data. Factor loadings were high (0.70-0.89), and Cronbach α was .89.

The purpose of this study was to identify the effect of attitudinal inoculation videos on the ability to identify misinformation, willingness to share misinformation, and willingness to vaccinate. We found there was a significant effect of having been exposed to an inoculation video compared to a control video across each outcome of interest. Our results support our initial hypotheses that inoculation messaging can increase the ability to identify misinformation, decrease willingness to share misinformation, and increase willingness to vaccinate.

Our study explored not only whether there is an effect of inoculating participants but also whether inoculating against a narrative/rhetorical strategy or against factual misinformation is more effective. We also tested a hybrid video that combined fact-based and narrative/rhetorical strategies within the inoculation message. Past studies into the relative efficacy of narrative/rhetoric compared to fact-based appeals have suggested that narrative approaches are more likely to be persuasive to viewers [32-34,37-39,55]. However, we did not find significant differences across the 3 active intervention groups, suggesting that the content of the intervention video (narrative/rhetoric, fact based, or hybrid) does not impact the effect of the intervention. This may be attributable to the nuanced differences in the scripts used, which might have been difficult to distinguish in such a short time frame. Additionally, because there was only 1 video for each video script strategy, it is not possible to say whether there may be differences in other videos that used these same strategies. It is even possible that the cinematographic and casting choices behind each video (ie, the production and the medical workers who narrated them) themselves made indirect narrative and rhetorical appeals to viewers. These appeals include, for example, clean sets, approachable body language, hopeful music, and other emotionally engaging features of the videos. This, in turn, might have produced a “flattening” effect, rendering the appeals of the videos more uniform.

The intervention videos were designed to protect people against being misled by flawed argumentation used in common online mis- and disinformation, such as conspiracy theories [57,58]. In practice, this means that watching a video with an inoculation message and a weakened microdose of manipulation techniques allows viewers to discern more readily subsequent misinformation that makes use of similar flawed argumentation techniques. In this study, we were able to achieve this in videos that were only 30 seconds in duration. This is critical because 30 seconds is consistent with short attention spans for online video consumption on social media and the length of many ads online, enabling such a video to be shown in ad slots. Prior inoculation studies have tested longer-form videos or text [49,51,58]. These findings for the effects of 30-second videos have implications for the ability to disseminate inoculation messages in social media ad slots.

Furthermore, online platforms are a viable place to disseminate these interventions to affect perceptions and behavior, because information on vaccine injury is viewed and spread on social media and exposure to online misinformation is associated with lower vaccination intentions [17,59,60]. Social media platforms that may hesitate at the prospect of hosting inoculation videos containing a microdose of misinformation might be reassured that in the proper context of an inoculation message, these microdoses are vital to the overall discrediting of misinformation and disinformation.

This study has limitations. First, the study sample was not representative of the general population but rather was more highly educated, mobile app users. Testing the effects of the intervention with a broader audience and determining whether there are differential effects among particular subsamples of the population will add to the understanding of the effect of these types of videos. Second, as described before, further investigation is needed to explore whether there may be meaningful differences in narrative versus fact versus hybrid models that could not be detected in this study due to the study design; however, in general, our results suggest that fact-based rhetorical strategies can be as effective as a narrative-rhetoric or hybrid approach.

As an infodemic management strategy, approaches that go beyond fact checking, and do not simply focus on countering 1 piece of content, may be valuable. We found that attitudinal inoculation in video-based messages may be an intervention strategy that can be used in designing public health messaging campaigns to counter mis- and disinformation. Online dissemination of these videos could be a viable strategy to increase vaccine uptake and can be tried more broadly. Videos that use attitudinal inoculation to combat COVID-19 vaccine misinformation should be tested with a broader audience beyond the United States and on social media platforms, such as YouTube and TikTok. More research is needed to understand how videos with attitudinal inoculation perform when individuals are in a typical information consumption environment and faced with competing demands for attention.