Remdesivir clinical trial & more

Anecdotal evidence of positive outcomes from the antiviral remdesivir (developed by Gilead Sciences Inc.) have emerged through compassionate use in the previous months, but we needed randomized controlled trials (RCTs) to judge efficacy, or how well the drug worked. A summary of results from a clinical trial in China was inadvertently posted by the World Health Organization last week and did not look too promising. You can read more from TIME. Today, April 29, 2020, a statement was released from the National Institute of Allergy and Infectious Diseases (NIAID) summarizing the preliminary results from the use of remdesivir at 68 international sites. The science and medicine publication, STAT, covered it here. We can’t see the entire methodology to assess the study and analysis limitations, but here is a first pass at what we do know.

Let’s begin with a statistics primer. When statisticians try to assess statistical differences between groups, we perform hypothesis testing. We determine a null hypothesis, which is a statement claiming no relationship between two measured phenomena (treatment and outcome) or between specified populations (the group receiving a drug and the group receiving placebo). We then consider an alternative hypothesis, which basically says there is, in fact, a difference between the two groups. We cannot prove the alternative hypothesis, but we can reject that the null hypothesis was likely to have produced the observed data. We then decide what false positive rate we are willing to live with. That means, are we willing to incorrectly reject our null hypothesis some percent of the time. We call this the alpha level, or significance level, and in standard practice it is 5%. Here is a video from Khan Academy if you want to learn more.

The output of many statistical methods is a p-value, or probability value, which is the probability of obtaining results at least as extreme as the observed results of a given experiment, assuming the null hypothesis is correct. A p-value of 0.05 means if we completed 100 identical experiments, we could see results this extreme or more extreme in 5 of the experiments simply due to chance. The American Statistical Association has an informative editorial on p-values.

In practice, we use the significance threshold to dichotomize our findings as significant (p-value < 0.05) or not significant (p-value ≥ 0.05). But the quantitative nature of the p-value really gives us evidence that exists on a spectrum of 0 to 1. For example, observations with a p-value of 0.04 and 0.00000004 are treated the same, as significant findings. But a p-value of 0.04 and 0.06, while much closer in magnitude, are considered significant and non-significant, respectively. There is a lot of debate within the scientific community about p-values, determining significance, etc. You can read more here, here, and here. It’s a big ol’ can of worms that you should be aware of, but we don’t need to open to understand the findings of the remdisivir trial.

The trial looked at two outcomes: recovery time and mortality (aka death). Those receiving the treatment had a median time to recovery of 11 days versus 15 days for those receiving the placebo. Recovery was defined as hospital discharge or a return to normal activity. A statistical test was done to compare the median recovery time between these two groups. The null hypothesis here is that there is no difference in the median recovery time between the treatment and control groups, and the alternate hypothesis is that there is a difference. If we set our alpha at the standard significance level of 0.05, then with the reported p-value being less than 0.001, we would reject our null hypothesis. We interpret this hypothesis testing to mean there is a difference between the treatment and control groups which is so great we would only expect to see it by random chance in less than 1 in 1000 experiments. We can feel pretty confident that the treatment is responsible for the 31% faster time to recovery observed in this sample. However, it is unclear to what extent this modest effect will change the course of the pandemic.

For mortality as an outcome, 8% of the treatment group died versus 11.6% of the placebo group. The p-value for comparison between groups is 0.059, so with the same significance level of 0.05, we would not be able to reject our null hypothesis. But this doesn’t mean we accept the null hypothesis that there isn’t a difference in mortality between the two groups. It just means we just haven’t amassed enough evidence to prove there is a difference within the false positive rate we decided would be acceptable (0.05). This could provide enough hope that there is a difference between groups to encourage further study, potentially in larger samples.

An independent data and safety monitoring board (DSMB) reviewed the data and shared the interim analysis with those running the study. The DSMB is a committee of experts responsible for reviewing clinical trial data to preserve study integrity. Because of this, we can be hopeful that no one with a vested interest in the pharmaceutical company’s fortunes or political incentives could sway the outcome of the clinical trial.

A formal report should be released in the coming weeks and we will know more about the study limitations. You can read Gilead’s statement too. The NY Times is reporting the FDA plans to announce as early as today the emergency use authorization for remdesivir. My personal take is this is obviously not the panacea we were all hoping for, and I hope the additional treatments in development or trials will prove more effective.

Here are additional news & views:

• A retrospective analysis (scientific analysis happens after events occur) of 368 patients seen through the US Veterans Health Administration suggest hydroxycholoroquine is not helpful as a COVID-19 treatment and may even be associated with worse outcomes. In a recent pre-print researchers showed that patients treated with hydroxychloroquine were more likely to have died than the control group (hazard ratio 2.61 with 95% CI 1.10-6.17), but there was not a significant association between treatment and death/survival for those receiving hydroxychloroquine and azithromycin (a common antibiotic). Randomized controlled trials are ongoing and will provide clear guidance on use of this drug. A correspondence was recently published in Nature Medicine described incidence of heart abnormalities in patients treated with hydroxychloroquine and azithromycin. In 84 patients the baseline average QT-interval, a measurement used to assess the electrical signals in the heart, was prolonged from 435±24 ms to 463±32 ms. In 9 patients (11% of cohort) the prolongation was severe to the point of putting them at risk of sudden cardiac death. This is one of the risks of using this drug as a potential treatment.

• Community transmission was occurring so early in the U.S. that a non-travel related COVID-19 death has been confirmed from February 6 in California. Read the press release from Santa Clara county and NY Times reporting. With this timestamp it’s clear that we were really shockingly behind at preventing this from spreading in the United States. This lines up with virologist Dr. Trevor Bedford’s work showing first U.S. introductions in January 2020 as explained here.

• The CDC released characteristics of patients hospitalized with confirmed COVID-19 in the 4 week period ending March 28, 2020. In the Morbidity and Mortality Weekly Report they describe 89.3% to have one or more underlying conditions with the most common being hypertension (49.7%) followed by obesity (48.3%) which are also fairly common co-morbidities in the United States. “Among the 1,482 laboratory-confirmed COVID-19–associated hospitalizations reported through COVID-NET, six (0.4%) each were patients aged 0–4 years and 5–17 years, 366 (24.7%) were aged 18–49 years, 461 (31.1%) were aged 50–64 years, and 643 (43.4%) were aged ≥65 years.”

• The NIH Director, Dr. Francis Collins, highlighted the challenge of tracking the virus’ spread on his blog. Traditional containment strategies are testing of symptomatic cases, contact tracing and quarantine. Due to the spread of disease before symptoms or even without symptoms, we need to expand the goals of testing to 1) help us identify asymptomatic people infected by SARS-CoV-2 to initiate quarantine so they cannot infect others 2) understand who has recovered from SARS-CoV-2 and may now possess immunity. The National Institute of Allergy and Infectious Disease (NIAID) released their strategic plan for COVID-19 research for the next four years. The main priorities are
  1) Improve knowledge about the virus and the disease
  2) Support development of diagnostics and assays (i.e. testing for the disease or antibodies)
  3) Characterize and test therapeutics to treat the disease
  4) Develop safe and effective vaccines against SARS-CoV-2

  In addition, the NIH recently announced a public/private partnership to speed COVID-19 vaccine and treatment options. The Accelerating COVID-19 Therapeutic Interventions and Vaccines (ACTIV) partnership will focus research and development efforts. The World Health Organization also announced worldwide collaborations to accelerate development of COVID-19 health technologies and ensure their equitable access across the globe.

• Facebook and Carnegie Mellon have collaborated on a survey and have released preliminary data as a COVID-19 Symptom map.