The Covid-19 pandemic has brought into sharp relief a problem that has been plaguing clinical research for decades: the unreliability of the entire clinical research “industry”.
The clinical research industry is composed of clinical researchers, medical research ethics committees, journal editors and journal peer reviewers. Together, this team is supposed to ensure that the end-user clinic doctor is given reliable research information to inform his clinical decision-making. This is referred to as evidence-based medicine. Because the end-user doctor has no access to the original data and cannot ordinarily request for further statistical tests, it falls upon this quadrumvirate to act as a quality control team. But they have repeatedly proven to be feckless and incompetent.
On June 4, we witnessed the embarrassing spectacle of two of the highest “impact factor” medical journals in the world retract a published study because of a lack of due diligence prior to publication. That is, two independent vetting systems, at the US-based New England Journal of Medicine (NEJM) and the UK-based Lancet, had failed. To make matters worse, the World Health Organisation (WHO) had previously rushed to suspend the hydroxychloroquine arm of the ongoing Solidarity clinical trials of drugs to treat Covid-19 because the retracted study had offered data suggesting patient harm.
It took readers just a few hours to do what the editors of the two journals and experts at WHO should have done, ie, check the “face value” of the study. WHO has restarted the trial since the retraction by the two journals – but it is now going to be very difficult to persuade patients to agree to participate in it.
Firstly, the clinical researcher. Prior to the 1970s clinical research was mainly “qualitative”, comprising case reports and case series. No special training was required to do or read this sort of research. But by the 1980s, clinical research became more “quantitative” with randomised controlled trials, cohort studies and diagnostic test accuracy studies, among others. This required special skill sets to carry out. It was a logical requirement that clinical researchers would now need comprehensive data science qualifications. Yet hardly any of our current crop of clinical researchers has such degrees. Why? Because research institutions do not want to spend the money to train their researchers. Even a cursory investigation shows that none of the authors of the retracted NEJM and Lancet publication seem to have such data science qualifications.
Secondly, the medical research ethics committees (MRECs). MRECs are guided by the Declaration of Helsinki, a statement of ethical principles for medical research made in 1964. The MRECs are supposed to be independent regulators. They are required to deliver usable research to guide patient care. They are supposed to oversee the research from start to finish. This requires training equivalent to or more specialised than that required of clinical researchers. But in reality that does not happen. Further, MREC members are mainly drawn from the ranks of salaried employees of the research institution concerned. Any signs of independence are mainly cosmetic in nature. It is unlikely that they can insist on important aspects such as data auditing without being replaced by their employer with more compliant employees. Without proper audits, there is no way of ensuring that data has not been fabricated. Research institutions do not want their steady stream of research publications to be interrupted by time-consuming data audits. Further, they might be saddled with the problem of disciplining senior staff if data audits show up irregularities.
Thirdly the journal editor. This person is one of the main lines of defence against scientific misinformation and disinformation. When a study comes in, the journal editor is supposed to go through it to see if it appears to be genuine research at face value. Clinical research data is difficult data because it is composed of extracted data (age, gender, height) and abstracted data (such as diagnosis and clinical findings). Data can be extracted electronically but abstraction needs an experienced physician to summarise and reconcile messy information in case notes. Nobody wants to be treated on the basis of data abstracted by an unqualified student. So the journal editor has to check the authors’ credentials. The journal editor will also have to check on the likelihood that the data is fabricated by looking into the details of the research institution. For example, a small district hospital is unlikely to have 100 dengue deaths within one year. Only the journal will have the resources to check on these issues. The journal editor also has to arbitrate in cases where authors and peer reviewers have serious disagreements. The editor may also have to write editorial commentary highlighting the methodological and statistical difficulties the study exhibits to alert the busy end-user reader.
So it follows that journal editors for clinical research must have data science degrees as well. The journal will also need a few editors who are statisticians to deal with more serious complaints. However, this is not usually the case. A quick look at our medical journals shows that qualified journal editors are a rarity.
Fourthly, the peer reviewer. The peer reviewer serves to independently decide if the clinical question posed by the research authors was answered using the correct research methodology and statistical analysis. So it logically follows that the peer reviewer should possess both a clinical specialist degree and a data science degree. Unfortunately, this is again a rarity.
The end result is a mess that passes for “clinical research”. These messes can cause death. It is high time the government put the whole clinical research industry directly under the Health Ministry and regulate it strictly for patient safety. We don’t need to be flooded with more low-quality research. We need better high-quality research. And there is no need for clinical research to be scattered among various research institutions. Malaysia has its own National Institutes of Health under the Health Ministry. It is their job to oversee and coordinate all clinical research in the country.
DR MANIMALAR SELVI NAICKER
Histopathologist and statistician
Subang Jaya, Selangor
Already a subscriber? Log in
Get 20% OFF The Star Digital Access
Cancel anytime. Ad-free. Unlimited access with perks.
