What VAERS Can and Can’t Do, and How Anti-Vaccination Groups Habitually Misuse Its Data

For decades, an unassuming government vaccine safety surveillance system has done its job, quickly flagging possible side effects and allowing scientists and regulators to investigate further. 

But for nearly as long, the ​​Vaccine Adverse Event Reporting System, or VAERS, has also been exploited by people opposed to vaccination. With a publicly searchable database, full of unverified reports of health problems that occurred sometime after vaccination, VAERS has proven irresistible to the anti-vaccination community, which often falsely claims the number of reported deaths or other issues is proof that vaccines are dangerous.

That’s despite the fact that the reports aren’t vetted for accuracy and don’t mean that a vaccine caused a particular problem.

VAERS is an early warning system used to identify potential safety concerns after a vaccine has been authorized or approved in the U.S. It’s often described as a “frontline” system, since it’s frequently the first vaccine safety system to detect a problem. But it’s also noisy and prone to distortion.

“Most of the anti-vaccine stuff that you hear, when they start to talk about how vaccines caused whatever, they’ll point to VAERS data,” Dr. Paul A. Offit, a vaccine expert at Children’s Hospital of Philadelphia, told us. “It is just manna from heaven to get bad information out there.”

While VAERS distortions were already a staple of vaccine misinformation prior to the pandemic, misuse of VAERS exploded with the arrival of the COVID-19 vaccines in late 2020. At FactCheck.org, we’ve written story after story debunking false or misleading claims about the COVID-19 vaccines that were based on misunderstandings about VAERS — and so have our fellow fact-checkers.

And now, one of the most notorious abusers of VAERS data is running for president. Robert F. Kennedy Jr., the nephew of assassinated President John F. Kennedy and a prominent anti-vaccine advocate, announced his campaign challenging President Joe Biden in April. (Kennedy has stated that he is for safer vaccines and is not “anti-vaccine,” but many of his arguments against vaccination are inaccurate or misleading and typical of the movement.)

In 2016, Kennedy founded a group that would become Children’s Health Defense, a nonprofit that traffics in anti-vaccine misinformation and disinformation. Hundreds of stories on Kennedy’s website mention VAERS.

Given the misuse and confusion around VAERS, a research team at the University of Pennsylvania’s Annenberg Public Policy Center — led by APPC Director Kathleen Hall Jamieson and in partnership with Critica Science — has proposed renaming VAERS “Vaccination Safety Monitor” or “Vaccination Safety Watch.” APPC is FactCheck.org’s parent organization.

Here, we’ll explain how VAERS works and run through five misconceptions that anti-vaccination activists wield to mislead people about vaccines.

A Frontline System, Ripe for Distortion

As we’ve explained before, vaccines given to the public have already been tested in clinical trials, but those trials can only be so big and aren’t expected to be able to identify rare side effects. That’s where VAERS and other post-marketing safety surveillance systems come in.

VAERS, which began in 1990 and is co-run by the Centers for Disease Control and Prevention and the Food and Drug Administration, collects reports of health problems that occur after vaccination. Anyone can submit a report, regardless of whether it’s likely the vaccine caused the event.

The CDC and FDA then review the reports in a variety of ways, and further investigate any possible safety concerns.

“VAERS is designed to detect unusual or unexpected patterns,” Dr. Tom Shimabukuro, director of the CDC’s Immunization Safety Office, told us in an interview. “It’s really about pattern recognition.”

Key strengths of VAERS are its large size and speed. Because VAERS reports draw from across the country, even a very rare event can be quickly identified as a possible side effect. 

Most famously, VAERS was the first system to raise concerns about a link between intussusception, a type of intestinal blockage, and RotaShield, the first rotavirus vaccine. In June 1999, just nine months after approval, 10 reports of intussusception had been reported to VAERS in infants who had received the RotaShield vaccine. This triggered further study of the issue and led CDC to temporarily suspend the shot the following month. The manufacturer recalled the vaccine a few months later, after other studies confirmed the safety signal.

Susan S. Ellenberg, a biostatistician at the University of Pennsylvania’s Perelman School of Medicine, told us the RotaShield example is the “poster child” for how VAERS can work.

VAERS has successfully flagged other safety concerns, including inflammation of the heart and surrounding tissue, known as myocarditis and pericarditis, which are the primary serious side effects of the mRNA COVID-19 vaccines. The conditions are rare after vaccination and are most frequent in young males after a second dose.

The system is also used to monitor the safety of different vaccine lots and to identify risk factors for developing certain vaccine side effects. VAERS data, for example, contributed to the decision to advise people with a severe immunodeficiency to avoid the RotaTeq and Rotarix rotavirus vaccines.

VAERS is unique in having its data available for anyone to access. In the early years, people had to file Freedom of Information Act requests to access the data. But in 2001, in the spirit of transparency, the agency posted the data online for download, a CDC spokesperson told us. In 2006, the data became searchable in an online tool.

Many of the features of VAERS, however, also make it susceptible to bad actors.

“The minute it was created, you could have argued that this was going to be misused, or at least misunderstood, because you’re asking people to understand the difference between causality and coincidence,” Offit said.

Ellenberg, who oversaw VAERS at FDA between 1993 and 2004, told us the system’s data was misused “from the very beginning.” She recalled one effort by the National Vaccine Information Center, a prominent anti-vaccination group, to use VAERS data to claim that certain vaccine lots, or what it called “hot lots,” were dangerous.

“They would look at VAERS and find vaccine lots that had the most reports associated with them and put them out there as those were potentially more toxic,” she said. “What the truth is, is that vaccine lots are variable sizes” and it’s completely normal for a vaccine lot with 100,000 doses to have more VAERS reports than one with 3,000. Lot sizes are proprietary information and therefore are not publicly available.

As PolitiFact has reported, the National Vaccine Information Center created its own VAERS search tool in 2003 that has become a favorite of anti-vaccination activists, fueling VAERS-based misinformation.

Federal officials have attempted to explain the limitations of VAERS and to discourage misinterpretations of the data, both in disclaimers on the website and in multiple academic articles.

As early as 1997, Ellenberg explained in a journal article that the way VAERS is designed, “sensitivity takes precedence over specificity; reporting of all serious events following vaccination is encouraged, inevitably resulting in large numbers of reports that do not represent vaccine-induced problems.”

“VAERS data must be interpreted with caution due to the inherent limitations of passive surveillance,” Shimabukuro and colleagues wrote in a 2015 article published in Vaccine, noting that VAERS is “primarily a safety signal detection and hypothesis generating system.”

“VAERS data interpreted alone or out of context can lead to erroneous conclusions about cause and effect as well as the risk of adverse events occurring following vaccination,” they added.

Claims involving VAERS have nevertheless figured prominently in anti-vaccine efforts to reduce the reach of a variety of vaccines, including the measles, mumps and rubella, and human papillomavirus vaccines.

With the COVID-19 vaccines, Ellenberg said the problem became “substantially worse.” Offit agreed that claims have “dramatically increased.” And anti-vaccine activists are using the tactics honed during the pandemic to apply them once again to other vaccines.

Common Patterns of Deceptions

1) Inappropriately Assuming Causality (And Accuracy)

Perhaps the biggest misunderstanding about VAERS is that the health issues described in the reports are not necessarily caused by the vaccine — and are often purely coincidental.

“Reports in VAERS simply represent something that happened after you got a vaccine. They don’t tell you the vaccine caused this,” Ellenberg said.

In some cases, it may be reasonable to assume the vaccine was the cause, such as some swelling on an arm just after a shot. But usually, Shimabukuro said, the information provided in a report isn’t enough to know whether a health problem was caused by a vaccine.

“Vaccines protect against a particular thing, a particular disease. They don’t protect against everything bad that might ever happen to you,” Ellenberg said. And so it’s inevitable that bad things will occur by chance right after a vaccine, even when they have nothing to do with the vaccine. 

People are encouraged to file a report for any significant health problem even if they don’t think a vaccine was the cause. Health care workers and vaccine manufacturers are also required to file certain reports, also regardless of the level of suspicion of a vaccine.

And yet, the internet is littered with examples of people incorrectly presenting VAERS reports as events caused by vaccines. Sometimes the health problems are explicitly and inaccurately called side effects or labeled “vaccine-caused.” (Side effects, which are also known as adverse reactions, are considered to be caused by a shot.) Posts will also assume causality, for example, when citing VAERS data to give a supposed number of “COVID vaccine deaths.”

Some posts correctly note that VAERS reports may not have been caused by vaccines, but still mislead by calling the reports “vaccine injuries” or suggesting they are indicative of an important health concern.

Part of the issue, Offit said, is the terminology, including the name of the Vaccine Adverse Event Reporting System. In scientific parlance, the term “adverse event” does not imply a causal connection. It simply means the event occurred after vaccination, so there’s a temporal association that could very well be coincidental. To most of the public, though, that nuance is lost.

“Its mere name gives it the imprimatur of a causal association and that’s not what it is,” Offit said of VAERS. “It’s misnamed.”

On top of that, people often incorrectly assume that the reports must be true because they are in a government database.

But as the VAERS website explains in a disclaimer, reports “may contain information that is incomplete, inaccurate, coincidental, or unverifiable.” Reports are not vetted before being included in the database.

In a now classic example, Dr. James R. Laidler, an anesthesiologist and autism advocate, said he filed a report in VAERS in the early 2000s that claimed “an influenza vaccine had turned me into The Hulk.” The report went into the database and was removed only after someone from VAERS contacted him, and after a discussion, asked if it could be deleted.

“If I had not agreed, the record would be there still,” Laidler wrote in a 2005 blog post, “showing that any claim can become part of the database, no matter how outrageous or improbable.”

That’s not to say that most VAERS reports are made-up. As we’ve written, the number of obviously false hoax reports is below 1%, and it’s illegal to file a false claim. But it’s not always clear when a report is fraudulent, and research has shown that litigation — even related to health issues that scientists know are not caused by vaccines — can drive up reporting.

2) Misunderstanding or Ignoring How VAERS Works with Other Systems

People opposed to vaccines often focus on VAERS to the exclusion of other vaccine safety systems — ignoring the fact that some of those systems are used to determine whether a possible safety signal from VAERS is indeed a problem.

As Dr. David Gorski, an editor of the blog Science-Based Medicine who has been debunking claims about vaccines for more than a decade, observed on Twitter, the reason these activists “fetishize #VAERS as the ‘definitive’ be-all and end-all of vaccine safety databases is because it is so easily distorted and weaponized.”

“VAERS at its best is a hypothesis-generating system,” Offit said. It’s all about signal detection — it’s not meant to be the final word on vaccine safety. And it doesn’t work in a vacuum.

“It’s important for people to know that VAERS is one of many complementary systems that CDC and FDA and other federal partners use to monitor vaccine safety,” Shimabukuro said.

Statistical methods are used to analyze VAERS reports to quickly pick up on any unusual patterns. “If a possible safety signal is found in VAERS, further analysis is performed with other safety systems, such as the CDC’s Vaccine Safety Datalink (VSD) and Clinical Immunization Safety Assessment (CISA) Project, or in the FDA BEST (Biologics Effectiveness and Safety) system,” the VAERS disclaimer explains. “These systems are less impacted by the limitations of spontaneous and voluntary reporting in VAERS and can better assess possible links between vaccination and adverse events.”

Indeed, while VAERS is a passive system, relying on people to submit reports, several of these systems are active, meaning they automatically collect information at regular intervals. And unlike VAERS, some of these systems offer a way of comparing outcomes to a control group. 

The Vaccine Safety Datalink, for example, draws on electronic health records from across the country and contains information about which vaccinations were given and when. The data are updated every week, and can be used to compare the rates of possible side effects in people who received a particular vaccine with a similar group of people who were not vaccinated.

The CDC and FDA use several quantitative methods to probe VAERS data for possible safety signals. This includes disproportionality analysis, which essentially checks to see whether the adverse events reported for one vaccine are significantly different from those reported for other vaccines, which could be indicative of a problem.

Ellenberg likens these approaches to looking for a needle in a haystack. “What these methods do is pull out clumps and then you look for needles in the clumps.” After further investigation, she said, most of them will turn out to be nothing.

Because the number of administered doses was known, regulators also performed an observed versus expected analysis for the COVID-19 vaccines, Shimabukuro said. If the observed rate approaches or exceeds the expected rate, he said, “that may be evidence of a potential safety problem that might require further investigation.”

Agency physicians also do a lot of case review to investigate possible problems.

Importantly, this slicing and dicing of VAERS data can only point to a possible issue — it’s not confirmation of one. 

“Just because you exceed a statistical threshold does not mean you have evidence of an increased risk or evidence of a causal association,” Shimabukuro said, adding that such data mining “findings” are not necessarily safety signals. “There can be other reasons for these findings or they can be spurious findings or in some cases, they can be things that we expect to find.”

VAERS, therefore, must be viewed in the larger context of how safety signals are identified. Insisting that only VAERS has the right answers is illogical and fundamentally misconstrues how vaccine safety surveillance works.

3) Improperly Comparing Vaccines

Much of the misinformation about the COVID-19 vaccines using VAERS has focused on improper comparisons between vaccines. Claim after claim alleges that because so many more VAERS reports have been filed for the COVID-19 vaccines than for other vaccines, it must mean that they are dangerous.

This line of argument, however, is faulty. As we’ve previously written, there are several reasons why reporting to VAERS increased for the COVID-19 vaccines — and it doesn’t mean that the vaccines aren’t safe. 

To start, a large number of COVID-19 vaccines were given out in a relatively short period of time, with more doses and priority given to older and more medically vulnerable people. The VAERS reporting requirements are also higher for the COVID-19 vaccines. Health care providers, for example, are required by law to report any vaccine administration error, any serious adverse event following vaccination, and any COVID-19 case that results in hospitalization or death. With other vaccines, providers are only required to report select adverse events. And the incredible amount of publicity and scrutiny of the new vaccines is arguably unprecedented in modern history.

“You really can’t compare what happened during COVID to what’s happened with other vaccines in the past,” Shimabukuro said.

The closest example, he said, is the rollout of an influenza vaccine during the H1N1 pandemic in 2009. With that vaccine, he added, there was also a large increase in the number of reports to VAERS, and public awareness “was nowhere near what it is for COVID-19.”

Shimabukuro noted that the phenomenon of a spike in reporting with a new vaccine, known as the Weber effect, is well documented.

And he added, the COVID-19 vaccines have been following the expected trajectory of the Weber effect quite closely, with very high reporting early on, followed by a peak and then a drop-off to a somewhat normalized level.

“The trend is very similar to what we see for other vaccines — other new vaccines, other pandemic vaccines,” Shimabukuro said, with the extreme attention on the COVID-19 pandemic “accentuating that overall trend.”

How are regulators so confident that the increased reporting in VAERS isn’t a safety concern? Because all of the data — including from VAERS, but also from all the other systems — consistently show that the COVID-19 vaccines have a good safety record.

“It’s data from multiple systems in the United States and data from other systems in other countries in Europe and in Canada and Israel, and really all over the globe,” Shimabukuro said.

Despite all the claims about COVID-19 vaccine-related deaths, VAERS data do not suggest that the vaccines increase mortality.

Of the COVID-19 vaccines ever offered in the U.S., only the Johnson & Johnson vaccine has been causally linked to thrombosis with thrombocytopenia syndrome, or TTS, which can be fatal. TTS is a blood clotting condition combined with low blood platelets and is extremely rare. Six reports of the condition to VAERS led regulators to temporarily suspend the use of the J&J vaccine in April 2021. Through May 2023, monitoring has identified nine deaths from TTS that are considered to be due to the vaccine. The J&J vaccine is no longer available in the U.S., after the last doses expired in May.

“There is no hiding in the world of vaccines when you vaccinate hundreds of thousands and then millions and tens of millions of people,” Offit said. If a vaccine is truly responsible for a serious side effect, he said, it will be apparent.

4) Exaggerating the Issue of Underreporting

Another common anti-vaccine talking point is that because people voluntarily report to VAERS, it invariably is an undercount of vaccine “harms.” Vaccine opponents often try to calculate how much underreporting exists and multiply the number of reports by certain factors to arrive at the “real” number of vaccine side effects. 

But this approach is flawed. It’s true that by design, VAERS can’t capture every side effect that is due to a vaccine. But it’s also the case that many of the health problems in VAERS aren’t caused by a vaccine.

“There’s underreporting and there’s overreporting,” Ellenberg said, referring to both scenarios. 

“The suspected adverse events are underreported. I think that’s probably true. But the keyword there is suspected — they’re not necessarily true, truly caused by vaccines,” Offit said, adding that that’s expected with a passive system. That’s precisely why other, active vaccine safety systems are also used to monitor vaccines. 

And there’s no simple way of determining how much underreporting exists. Anti-vaccine groups commonly cite a 2010 report from Harvard Pilgrim Health Care that stated “fewer than 1% of vaccine adverse events are reported.” 

But Dr. Michael Klompas, a public health surveillance researcher at Harvard Medical School and one of the authors of the report, told us in an email that the 1% number “takes into account that many adverse effects of vaccines are mild and expected so not worth reporting (sore arm, fatigue, local redness, etc.).”

Other researchers have attempted to estimate what’s called the reporting efficiency, or reporting sensitivity, of certain adverse events in VAERS, generally finding that the system more completely collects serious adverse events than mild ones.

An early effort in 1995, for example, found that VAERS detected 68% of vaccine-associated polio cases following the oral polio vaccine, but less than 1% of rashes after the MMR vaccine. (The oral polio vaccine has since been replaced in the U.S. with an injected vaccine that cannot give people the disease.)

Other work has found that for anaphylaxis, a potentially life-threatening allergic reaction that occurs rarely with any vaccine, VAERS captured anywhere from 13% to 76% of cases, depending on the vaccine. Another study estimated that VAERS caught 47% of cases of intussusception after the RotaShield vaccine. 

But as that paper noted, “Although the reporting completeness of VAERS has been evaluated for some specific vaccine-event associations, this information cannot be generalized.”

“The magnitude of underreporting varies widely, depending upon factors such as the severity of the event, proximity in time of the event to vaccination, and preexisting awareness on the possible association of the event to the vaccine,” it reads.

While underreporting is a legitimate limitation of VAERS, the system is not intended to capture everything. And applying ad hoc estimates for underreporting, particularly to all adverse events, or for adverse events that have not been linked to vaccination, is scientifically unsound and misleading.

5) Incorrectly Assuming All Reports Are Serious

Finally, another misconception is the incorrect notion that all reports in VAERS are serious. Again, part of this hinges on the use of technical language. “Adverse event” sounds serious to many people, but it includes minor incidents, such as a sore arm. 

Less than 10% to 15% of U.S. reports in VAERS are considered “serious” — a regulatory term that means the event was life-threatening or involved hospitalization, prolonged hospitalization if someone was already hospitalized, persistent disability, a birth defect, death, or required medical attention to prevent one of these outcomes.

The CDC requests follow-up information for all serious reports, which, like their non-serious counterparts, may be entirely coincidental. As the CDC explains, while serious events happen after vaccination, “they are rarely caused by the vaccine.”

The non-serious and serious classification isn’t perfect. “Some degree of misclassification is inherent,” a 2004 review by government scientists explains, noting that injection site reactions typically are “not of great clinical significance but may be classified as serious if they result in a brief hospitalization.” On the other hand, something like Bell’s palsy, a usually temporary facial paralysis, is medically important, but may not be classified as serious because it involves outpatient care.

Still, it’s clear that many of the health issues reported to VAERS — which again, are not necessarily caused by vaccines — are relatively minor, and people who like to highlight the sheer number of reports to suggest vaccines are dangerous are not being fully transparent.

Imperfect, But Still Necessary

For all of its limitations and susceptibility to distortion, experts generally told us they thought VAERS served an important role.

Ellenberg, for example, said she thought VAERS could be the fastest way to identify a vaccine safety problem. 

Offit, however, was less sure of its utility.

“I would argue that because it’s so massively misused and massively misunderstood,” which has caused “a lot of people to choose not to get a vaccine,” he said, “I think it has done far more harm than good.”

Still, he doesn’t think VAERS should go away. Rather, he thinks VAERS should not be made publicly available. That would limit the misinformation, but still allow the system to do its job.

Putting the genie back in the bottle, though, may be impossible. And for now, the CDC doesn’t agree.

“We understand that there is the potential for misuse and misrepresentation of VAERS data,” Shimabukuro said. “However, we think the benefits of being transparent and providing these data as a public service outweigh the potential harms.”

---

Editor’s note: SciCheck’s articles providing accurate health information and correcting health misinformation are made possible by a grant from the Robert Wood Johnson Foundation. The foundation has no control over FactCheck.org’s editorial decisions, and the views expressed in our articles do not necessarily reflect the views of the foundation.

The post What VAERS Can and Can’t Do, and How Anti-Vaccination Groups Habitually Misuse Its Data appeared first on FactCheck.org.