Could clinical trials contain falsified data?

An investigation has been launched into some clinical trials as statistical patterns arouse suspicion over reliability. A large number of trials may contain falsified or incorrect data, according to a recent review of published papers.

How the analysis was carried out

Statistical tools

Consultant anaesthetist, John Carlisle, fromTorbay Hospital, undertook in-depth analysis. He reviewed data collected from over 5,000 clinical trials, which had been published over a period of fifteen years in the New England Journal of Medicine, Jama and six other journals specialising in anaesthesiology. He found a total of 90 trials which contained suspect statistical patterns. No matter how innocent, these patterns were unlikely to have appeared by chance alone, and this rate of error is concerning, especially considering results play a vital role in deciding the treatments paths of certain patients.

The statistical tools used to determine these results compared baseline data sets, including blood pressure, weight, gender and height, against known distributions of these different variables in random population samples. Significant variations from the expected data is an indication of data tampering, error or complete fabrication.

What happens next for flagged trials

Serious allegations

In contrast to many other studies of this nature, Carlisle included as part of the review a list of the trials he identified, singling out authors and exposing them to potential criticism. Editors of the various journals involved now intend to approach associated authors and trigger more involved investigations where necessary.

The journal Anaesthesia published results of this latest analysis and editor-in-chief, Dr Andrew Klein, has called for the urgent review of studies identified within the report. He has also called for the adoption of similar methods of screening by all journals, as well as part of the entire trial process, including adaptive phase 1 studies (http://www.richmondpharmacology.com/adaptive-phase-i-studies.php).

Caution around interpreting the findings

Public Understanding of Risk Professor, Sir David Spiegelhalter, at the University of Cambridge, feels the publications must be scrutinised more closely. However, he has suggested caution when it comes to understanding the cause of such discrepancies at this stage.

The New England Journal of Medicine only accessed the cited papers upon publication of the report, however, they are carefully reviewing all information and consider the issue very serious. Jamal editor-in-chief, Howard Bauchner, will also be undertaking a full validity assessment of the allegations before determining next steps, including getting in touch with authors to give them the opportunity to respond to the allegations.

Summary

This article reports on an investigation by anaesthetist John Carlisle into more than 5,000 published clinical trials, which found 90 with suspicious statistical patterns suggesting possible data tampering, error or fabrication. It covers how journal editors plan to investigate flagged trials and contact authors, while one expert urges caution about jumping to conclusions on the cause of the discrepancies.

Questions this article raises

• What ultimately happened to the 90 trials flagged by this analysis?
• How common is data fabrication found to be across published medical research generally?
• What statistical method did Carlisle use to detect suspicious patterns?