Quantitative errors in the Cochrane review

Link: https://www.researchsquare.com/article/rs-3486610/v1

Summary of the highlights of the paper

Quantitative errors in the Cochrane review on “Physical interventions to interrupt or reduce the spread of respiratory viruses”

Yaneer Bar-Yam^1,2, Jonathan M. Samet³,  Alexander F. Siegenfeld^1,4, Nassim N. Taleb⁵

¹New England Complex Systems Institute, ²World Health Network, ³Colorado School of Public Health, ⁴MIT, ⁵Tandon School of Engineering, New York University, and Universa Investments

The COVID-19 pandemic has brought a heightened sense of urgency in the scientific community regarding the need to advance understanding and prevention of pathogen transmission, particularly concerning infectious airborne particles and the utility of various preventive strategies in reducing the risk of infection. There are extensive studies validating scientific understanding about the behavior of larger (droplets) and smaller (aerosols) particles in disease transmission and the dosimetry of particles in the respiratory track. Similarly, modalities for respiratory protection against particles in the size range spanned by infectious particles, such as N95 respirators, are available and known to be efficacious with tested standards for harm reduction across environments including physical, chemical and biological hazards. Even though multiple studies also confirm their protective effect when adopted in healthcare and public settings for infection prevention, overall, studies of protocols of their adoption over the last several decades in both clinical trials and observational studies have not provided as clear an understanding. Here we demonstrate that these studies are strongly biased towards the null by infections resulting from transmission outside of the investigated environments and study participants. Such study limitations are frequently mis-stated as not influencing the conclusions of research on respiratory protection. The reason for the failure to properly analyze the studies is that the standard analytical equations used do not correctly represent the random variables that play a role in the study results. By correcting the mathematical representation and the equations that result from them, we demonstrate that conclusions drawn from these studies are strongly biased and much more uncertain than is acknowledged, providing almost no useful information. Even with all these limitations, we show that existing results, when outcome measures are properly analyzed, consistently point to the benefit of precautionary measures such as N95 respirators over medical masks, and masking over its absence. We also show that correcting manifest errors of widely reported meta-analyses also leads to statistically significant estimates. Our results have implications for the design of studies and analyses on the effectiveness of respiratory protection and on using existing evidence for policy guidelines for infection control.

New paper on Masks:

Some claim scientific studies show masks don’t work, or that surgical masks are as good as N95s. Just released paper shows this claim is false. Masks work, and N95 respirators give a much higher benefit than surgical masks.

The controversy about N95 respirator and mask efficacy is based on claims that studies (RCTs and meta-analyses) have no observed effect. Our paper shows these claims result from the use of incorrect mathematical equations.

The incorrect mathematical equations bias the results. Reanalyzing the data with the right equations show that masks, and N95s in particular, do prevent transmission.

Here are a few highlights of the paper.

1. Studies comparing N95s and surgical masks in healthcare workers ignore infections at home—outside study protocols. This effect is so large, a study claiming N95s give no more than 2X improvement, should instead say no more than 10X improvement, or even higher.
2. The study also did not measure how much healthcare worker masking affects patients. Including patients would result in about 2X higher benefit of N95s in preventing infections and even more for reducing harm because of the higher vulnerability of patients.
3. The study also did not consider two way masking which would bring the improvement by 10X up to 100X = 10X * 10X due to both the infected individual (source), and the individual who is not infected, wearing an N95 respirator.
4. While the studies focus on statistical significance, if we look at the benefit from improved masking, the meta-analyses show there is a high probability of benefit ratio, such as 15X for N95s compared to surgicals.
5. If we exclude from the meta-analyses one incorrectly designed study that specified masking within 6 ft of patients (not correct for airborne transmission) then results show there is statistical significance—undermining the claim of no observed effect.

Overall, when the equations are corrected, the evidence shows N95s compared to surgical masks, and masks compared to no masks, improve health outcomes by preventing transmission. This is consistent with what is known from other studies about airborne transmission.9/

Link: https://www.researchsquare.com/article/rs-3486610/v1