In a recent study , a number of leading researchers, including Professor Riccardo Polosa, have unpicked scientific flaws in popular vaping studies. They chose the ones most often cited by researchers and policy makers. When a study is “cited”, it means another academic article has used it as a reference. This generally means scientists consider the research credible. This study raises a question of whether that is a reasonable assumption in the field of vaping and e-cigarettes.
Their findings confirm something we have said on many occasions. The vast majority of popular vaping studies have their basis in flawed methods. The researchers analysed the 24 most-cited pieces of vaping research. They determined the studies’ popularity using a Google algorithm. This helped them to find the most frequently referenced journal articles on vaping.
The papers they look at related to health outcomes (4 papers), smoking initiation (11 papers) and smoking cessation/reduction (10 papers). The conclusions of almost all the articles interpreted data in a very similar, but incorrect ‘anti-vaping’ theme.
Unfortunately, the publication of these popular vaping studies often leads to articles in mainstream media. Examples of headlines include “Vaping associated with an increased chance of heart attacks” and “Vaping is a gateway into smoking”.
These are hard-hitting statements. They certainly deter smokers from trying vaping. But is the science behind the most popular vaping studies reliable enough to back up these damaging headlines?
How accurate is the science in these anti-vaping studies?
The researchers found serious flaws in nearly all of the articles in question. Professor Polosa said, “I’m astounded that such low-quality studies have made it through editorial review in prestigious scientific journals.” The results of the team’s review are overwhelming and expose the poor science often associated with vaping research.
The key flaws discussed included a ‘lack of clear hypothesis, ‘failure to control for confounding factors’ and ‘poorly identified outcome measures’. Many of the studies were cross-sectional, which can be particularly unreliable with regard to vaping. This type of study is a snapshot taken at one moment in time. That means that any true analysis must properly account for previous illness and habits. In the case of vapers, the fact that we are almost all ex-smokers is the startling factor most often ignored.
In the ‘Analysis of common methodological flaws’ research, the authors also exposed sloppy research into the “Gateway theory”. This theory suggests that if young people begin vaping, it leads them on to smoking. It is a common anti-vaping scare story, and is particularly popular in the USA. It has led to the introduction of various damaging pieces of legislation in areas of the USA. However, Professor Polosa et al, describe the body of literature on this topic as “particularly unreliable”. The researchers stated, “Overall, the results and discussion contained numerous unreliable assertions due to poor methods, including data collection that lacked relevance, and assertions that were unfounded”.
Recommendations for improved vaping research
Arguably, there is one thing most concerning about the studies in question.The most respected scientific journals have published them. This means that when cited, they carry credibility, despite their flaws. Therefore the researchers who have uncovered these flaws have suggested a number of improvements.
These suggestions were specific to each type of study (Health, smoking initiation, smoking cessation). One of the standout improvements related to confounding factors and historical habits. Basically speaking, vaping researchers need to account correctly for participants’ previous smoking history. Particularly with health outcomes, a vaper’s previous smoking habit will have caused much of the damage. Researchers often fail to reflect this. This means they incorrectly imply vaping is a cause of poor health outcomes. A prime example of this limitation was seen in a retracted Stanton Glantz article . The publishing journal forcibly retracted this fundamentally flawed research without the author’s agreement. It had used data on heart attacks. However it did so without considering an important factor. Had the patients started vaping before or after the heart attack?
Other suggested improvements included the discussion of bias in the sample set, and avoiding the assumption of causal relationships without evidence. All of these improvements should already be common practice for the scientists behind the studies. Nevertheless, respected journals still continue to publish them with genuinely damaging consequences.
What is the result of poor vaping research?
Poor vaping research can have a significant impact on public health. The studies in question are the most highly-cited pieces by journalists.
The articles themselves very rarely outline the limitations of the research behind the headlines. Of course many people never read beyond the headline anyway. This negatively affects smokers’ and vapers’ perception of vaping harm. According to Action and Smoking and Health (ASH) , “Nearly a third of smokers incorrectly believe vaping is more or equally as harmful as smoking”. In the US things are even worse, with just 2.6% of adults able to correctly identify that vaping is much less harmful than smoking . 8.6% give the “nearly correct” answer of vaping being less harmful, but that leaves 9 out of 10 US adults not knowing that vaping is safer than smoking. A public health disaster!
This misinformation undoubtedly discourages some smokers from attempting to quit with an e-cigarette. Vaping is proven to be the UK’s most popular and effective quit aid. This means incorrect perception of vaping harm is a huge missed opportunity for public health. With credible research, more smokers will have an accurate understanding of vaping’s very low harm level compared to smoking. This in turn will result in more smokers trying vaping, and smoking rates will without doubt go down.
As Professor Riccardo Polosa commented on the study’s conclusions, “The need for unbiased research is urgent”.
 C.Hajat et al. Analysis of common methodological flaws in the highest cited e-cigarette epidemiology research. Internal and Emergency Medicine. 2022. [Online] Available from: https://link.springer.com/article/10.1007/s11739-022-02967-1
 S.Glantz et al. Retraction to: Electronic Cigarette Use and Myocardial Infarction Among Adults in the US Population Assessment of Tobacco and Health. Journal of the American Heart Association. 2020. [Online] Available from: https://www.ahajournals.org/doi/10.1161/JAHA.119.014519
 Use of e-cigarettes (vapes) among adults in Great Britain. Action on Smoking and Health. 2021. [Online] Available from: https://ash.org.uk/wp-content/uploads/2021/06/Use-of-e-cigarettes-vapes-among-adults-in-Great-Britain-2021.pdf
 C. Bates. E-cigarette risk perceptions – An American crime scene. The Counterfactual. 2022. [Online] Available from: https://clivebates.com/e-cigarette-risk-perceptions-an-american-crime-scene/