The LSTM Seminar Series continued this week with a presentation fromDr Emily Sena of the Centre for Clinical Brain Sciences, University of Edinburgh. Her seminar “Translational medicine: the new paradigm of systematic reviews of pre-clinical studies” was introduced by LSTM’s Professor Paul Garner.
Dr Sena has been working for some years on animal studies used to study stroke. This work is part of a larger group coordinated from Edinburgh and Melbourne, the Collaborative Approach to Meta- Analysis and Review of Animal Data from Experimental Studies or CAMARADES. This group have a data repository containing 14 disease areas, 4,500 studies, 15,000 in vivo experiments from over 120,000 animals, and carries out meta-analysis to examine what the combined data help show. This is part of an important process in making explicit the evidence base for moving forward into trials in humans, so called “translational research”.
This process has led the group to take a long hard look at the literature. It has become clear that many studies are underpowered, and there is such an academic pressure to publish positive results that authors don’t publish studies that don’t show anything. The resulting meta-analysis is then biased, suggesting a much larger effect than actually exists.
In addition, the studies are often poorly designed and reported, and this adds to the biases and research that is unreliable.
Part of the reasons for this is the strong drivers in life sciences incentives are about publication, funding and promotion so there is pressure to produce positive results with little attention to their validity. Not that this is unique to animal studies, it’s just not been investigated before.
Ultimately then this poor quality science can lead to squandering of resources on poor quality research that cannot be relied on.
Translational research not only needs reliable studies, but construct validity of experiments: that is, how well the animal studies mimic the conditions in humans, Dr Sena used the example of studies into the drug NXY-059 in experimental stroke where while 77% of patients in the clinical trial were hypertensive only 7% of the animals used showed those symptoms. She also used the example of drugs being tested for Alzheimer’s disease that were tested on mice without symptoms of the disease, so trials are therefore more about how to prevent rather than cure the disease.
Dr Sena concluded by looking at what needed to be done to address the problems of experiments being conducted without sufficient rigour, not all outcomes being reported and a priori analyses not always being reported. She called for protocols to be published, for there to be registries of experiments and posed questions about the possibility of reporting guidelines to ensure that preclinical trials can be more reliable and that their results are more likely to be translated with greater success to the clinical trial phase.