A study in Science in May 2024 explores the link between social media misinformation and vaccination intentions in the US. The researchers consider misinformation flagged and unflagged by fact-checkers on Facebook in relation to US COVID-19 vaccine hesitancy. The authors define the “impact” of misinformation as a combination of “exposure and persuasive influence” as they seek to quantify the impact of misinformation on Facebook. They find that, while exposure to fact-checked misinformation can cause vaccine hesitancy, the degree to which a story “implies health risks from vaccines” best predicts negative persuasive influence. Furthermore, content that suggested that the vaccine was harmful to health reduced vaccination intentions.
The infodemic
The authors recognise the spread of misinformation online as a “key concern” for policymakers and the public, as well as a “major focus of study” for researchers.
“This attention is largely motivated by the assumption that misinformation causes substantial real-world harm.”
However, this attention has been “largely relegated to assertions”. This has created a “gap” that is relevant to COVID-19 vaccine misinformation, and although the term “infodemic” is “frequently cited as an obstacle to the adoption of public health measures”, there has been little done to show a causal connection.
A framework for Facebook
The paper presents a framework for estimating causal impact at scale and applies the approach to quantify the “harm” caused by COVID-19 vaccine misinformation on Facebook. First, the authors consider what would be necessary for online misinformation to “have the sweeping societal impact so broadly ascribed to it”. They suggest that for any information to have a “widespread impact on people’s behaviour” it must be seen, and by a “large” number of people.
“Thus, impact arises from the interaction between exposure and persuasive influence.”
The approach that is proposed combines:
- Results from experiments measuring the effect of different vaccine-related headlines on vaccination intentions
- Data about the exposure to vaccine-related URLs on Facebook
All popular vaccine-related URL content on Facebook was included, not just content that was flagged by fact-checkers, but they contrast misinformation with “vaccine-sceptical” content.
Content informing behaviour
The research considered “which types of vaccine content changed willingness to take a COVID-19 vaccine, conditional on exposure”, through two large-scale online survey experiments. The approach measured behavioural intentions and assessed differences in persuasive effects across messages.
In the first study, 8,603 American participants on Lucid, an online survey platform, were shown a neutral control post or single piece of vaccine misinformation from a bank of 40 pieces of content that had been debunked by fact-checkers. Participants were asked to answer a set of questions about the willingness to take a COVID-19 vaccine before and after exposure.
“Consistent with conventional wisdom, we found that exposure to a single piece of vaccine misinformation decreased vaccination intentions by 1.5 percentage points on average.”
While the effect did not vary significantly on the basis of participants pretreatment vaccination intentions, gender, age, political party, or vaccine status, it varied “substantially” across different pieces of misinformation.
“An item did not lower vaccination intentions simply by virtue of being false, which suggests that other dimensions of the content were relevant beyond veracity.”
The second study involved a collection of 90 “highly shared” vaccine-related articles from Facebook. 10,122 American participants were involved in this attempt to measure the causal effect of each piece of content on vaccination intentions through the same procedure as the previous study. A new set of raters was presented to quantify content dimensions, labelling headlines on whether they were:
- Surprising
- Plausible
- Favourable to Democrats or Republicans
- Familiar
- Whether the item suggested that the vaccine was harmful versus helpful to a person’s health
The researchers ran a random-effects meta-regression predicting the treatment effect of each headline on vaccination intentions and found that the “only content dimension that consistently predicted a headline’s effect on vaccination intentions” was the extent to which a headline suggested the vaccine was harmful to a person’s health.
Interestingly, there was “no significant effect” on vaccination intentions from the source of the headline, contrasting a “low-quality domain” with a “mainstream domain”. Falsity was associated with a “more negative effect on vaccination intentions”, but when predicting treatment effect size using veracity and the extent to which a headline suggested the vaccine was harmful found the latter “remained significant”, whereas did not.
“These results indicate that suggesting the vaccine was harmful to health reduced vaccination intentions, irrespective of any potential effect of whether the headline was factually inaccurate.”
Information on Facebook
The paper then considers levels of exposure to vaccine-related content on Facebook, using the large-scale Social Science One dataset that was released by Facebook. The authors identified 13,206 URLs about the COVID-19 vaccine that were shared publicly >100 times on Facebook and published during the initial rollout period for the vaccine in the US: the first 3 months of 2021.
They found that URLs flagged by professional fact-checkers as false, out-of-context, or a mixture (flagged misinformation) received 8.7 million views. These accounted for “only 0.3% of the 2.7 billion vaccine-related URL views during this time. Content from “low credibility” domains received only 5.1% of views.
“Thus, exposure to flagged URL misinformation about vaccines on Facebook was relatively infrequent, owing to some combination of low baseline user viewership and explicit downranking by Facebook.”
However, the authors note that “even content not flagged by fact-checkers may have negative effects on vaccination intentions”. This is described as “vaccine-sceptical” content. Stories that did not contain “intentional falsehoods” but had implications that the vaccine might be harmful to health, did not face the same “scrutiny” as “outright falsehoods or content from low-credibility sources”.
“We find that exposure to vaccine-sceptical content far outstripped exposure to flagged misinformation.”
Lowered vaccine intentions
The authors estimate that the combination of flagged misinformation and unflagged vaccine-sceptical content lowered US vaccination intentions by “2.3 percentage points per Facebook user”. However, this effect was “almost entirely” by vaccine-sceptical content from mainstream sites that was not flagged by fact-checkers, rather than “outright false content published by fringe outlets”. This means that veracity-oriented interventions are “unlikely” to have reduced the spread of the content that had the “most overall negative impact”: unflagged vaccine-sceptical stories, often from mainstream outlets.
“Had exposure to this content been prevented, we estimate that vaccination intentions could have been 2.3 percentage points higher on average among Facebook’s 233 million US users – translating into ~3 million more vaccinated Americans.”
It is important to note that the data included only URL link content, which excluded information about native video, photo, or text-only content. Therefore, the finding is a “lower bound” of the total.
Policy implications
The results have “important policy implications” and highlight the need to consider “reach and impact” as well as veracity. In particular, the “grey area” content that is “misleading without being factually inaccurate” has potential to “inflict substantial societal harm”. Therefore, the authors call for a move beyond a “narrow focus on veracity” to “understanding, tracking, and potentially intervening on harmful content that is misleading without being literally false”.
Check out the full article here and don’t forget to get your tickets to join important discussions about vaccine misinformation at the Congress in Barcelona. For the latest vaccine insights and updates, why not subscribe to our weekly newsletters here?
