Canada Spends Logo

Explore our Budget 2026 analysis

Grant: Undergraduate Student Research Awards (USRA)

University of Toronto

$6,000.00

Estimating Publication Bias and False Discovery Risk in Clinical Trials

FY 2025-26

Summary
"Since Ioannidis (2005) declared that most published results in medicine - and other sciences - are false, researchers have wondered how many published results are false-positives. Conducting science-wide replication studies to answer this question is impractical. Statistical methods to estimate the false discovery risk often rely on untestable assumptions, making it difficult to study this question empirically. Another problem is that published results are often selected for significance. To overcome these problems, my colleagues and I have developed a statistical method that uses few assumptions and applied this method to abstracts of clinical trials in leading medical journals (Schimmack & Bartos, 2023). We found evidence of publication bias, but the false discovery risk was relatively low (13%) and could be reduced to less than 5% by using .01 as a criterion for statistical significance. A major limitation of this work was that the data were based on automatically extracted results reported in abstracts, which may not be representative of results in the actual article. Moreover, p-values were often not reported exactly. To overcome these limitations, I started a project that uses hand-coding of research articles that uses exact statistical results from the results section of articles. Furthermore, the project compares these results to the meta-analytic results in Cochrane Reviews, rigorous meta-analyses of clinical trials."
Program
Undergraduate Student Research Awards (USRA)
Location
Ontario, Canada
Recipient Type
Academia
Type
Grant
Source
CIHR Funding Decisions Database

As published

Id
531010
Name
El Huzaiel Abid Malak
Region
Ontario
Country
Canada
Orgtype
Institution Paid; Research Institution
Prcname2
Special Cases - Awards Programs
Projecttitle
Estimating Publication Bias and False Discovery Risk in Clinical Trials
Programtype2
Miscellaneous Programs
Orgnameinp2
University of Toronto
Cihramount2
$6,000
Cihrequipment2
$0
Programname2
Undergraduate Student Research Awards (USRA)
Keyworddelim
Articles; Assumptions; False Discovery; False-Positives; Hand-Coding; Medical Journals; Statistical Methods
Abstract
"Since Ioannidis (2005) declared that most published results in medicine - and other sciences - are false, researchers have wondered how many published results are false-positives. Conducting science-wide replication studies to answer this question is impractical. Statistical methods to estimate the false discovery risk often rely on untestable assumptions, making it difficult to study this question empirically. Another problem is that published results are often selected for significance. To overcome these problems, my colleagues and I have developed a statistical method that uses few assumptions and applied this method to abstracts of clinical trials in leading medical journals (Schimmack & Bartos, 2023). We found evidence of publication bias, but the false discovery risk was relatively low (13%) and could be reduced to less than 5% by using .01 as a criterion for statistical significance. A major limitation of this work was that the data were based on automatically extracted results reported in abstracts, which may not be representative of results in the actual article. Moreover, p-values were often not reported exactly. To overcome these limitations, I started a project that uses hand-coding of research articles that uses exact statistical results from the results section of articles. Furthermore, the project compares these results to the meta-analytic results in Cochrane Reviews, rigorous meta-analyses of clinical trials."
Competitiondate
202505
Approvedterm2
0 yr 4 mths
Pinamesdelim
El Huzaiel Abid, Malak
Approvedterm
0 an 4 mois
Cihrcontribution2
~ $6,000~ $0~