Selective Cutoff Reporting in Studies of Diagnostic Test Accuracy: A Comparison of Conventional and Individual-Patient-Data Meta-Analyses of the Patient Health Questionnaire-9 Depression Screening Tool.

Am J Epidemiol 2017 05;185(10):954-964

In studies of diagnostic test accuracy, authors sometimes report results only for a range of cutoff points around data-driven "optimal" cutoffs. We assessed selective cutoff reporting in studies of the diagnostic accuracy of the Patient Health Questionnaire-9 (PHQ-9) depression screening tool. We compared conventional meta-analysis of published results only with individual-patient-data meta-analysis of results derived from all cutoff points, using data from 13 of 16 studies published during 2004-2009 that were included in a published conventional meta-analysis. For the "standard" PHQ-9 cutoff of 10, accuracy results had been published by 11 of the studies. For all other relevant cutoffs, 3-6 studies published accuracy results. For all cutoffs examined, specificity estimates in conventional and individual-patient-data meta-analyses were within 1% of each other. Sensitivity estimates were similar for the cutoff of 10 but differed by 5%-15% for other cutoffs. In samples where the PHQ-9 was poorly sensitive at the standard cutoff, authors tended to report results for lower cutoffs that yielded optimal results. When the PHQ-9 was highly sensitive, authors more often reported results for higher cutoffs. Consequently, in the conventional meta-analysis, sensitivity increased as cutoff severity increased across part of the cutoff range-an impossibility if all data are analyzed. In sum, selective reporting by primary study authors of only results from cutoffs that perform well in their study can bias accuracy estimates in meta-analyses of published results.

Download full-text PDF

Source
http://dx.doi.org/10.1093/aje/kww191DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC5430941PMC
May 2017

Publication Analysis

Top Keywords

conventional meta-analysis
12
studies diagnostic
12
patient health
8
studies published
8
individual-patient-data meta-analyses
8
depression screening
8
screening tool
8
cutoff points
8
cutoff
8
increased cutoff
8
conventional individual-patient-data
8
health questionnaire-9
8
diagnostic test
8
cutoff reporting
8
selective cutoff
8
test accuracy
8
reporting studies
8
cutoffs
7
studies
6
published
6

Similar Publications