Am J Epidemiol 2019 Apr 17. Epub 2019 Apr 17.
Center for Population Health Sciences, Department of Medicine, Stanford University, Palo Alto, California.
Matching methods are assumed to reduce the likelihood of a biased inference compared to ordinary least squares regression. Using simulations, we compare inferences from propensity score matching, coarsened exact matching, and un-matched covariate-adjusted ordinary least squares regression (OLS) to identify which methods, in which scenarios, produced unbiased inferences at the expected type I error rate of 5%. We simulated multiple datasets and systematically varied common support, discontinuities in the exposure and / or outcome, exposure prevalence, and analytic model misspecification. Matching inferences were often biased compared to OLS, particularly when common support was poor; when analysis models were correctly specified and common support was poor, the type I error rate was 1.6% for propensity score matching (statistically inefficient), 18.2% for coarsened exact matching (high), and 4.8% for OLS (expected). Our results suggest when estimates from matching and OLS are similar (i.e. confidence intervals overlap), OLS inferences are unbiased more often than matching inferences, however, when estimates from matching and OLS are dissimilar (i.e. confidence intervals do not overlap), matching inferences are unbiased more often than OLS inferences. This empirical 'rule of thumb' may help applied researchers identify situations when OLS inferences may be unbiased compared to matching inferences.