Correcting the predictive validity of a selection test for the effect of indirect range restriction.

Authors:
Stefan Zimmermann
Stefan Zimmermann
University of Heidelberg
Germany
Dietrich Klusmann
Dietrich Klusmann
Universitätsklinikum Hamburg-Eppendorf
Germany
Wolfgang Hampe
Wolfgang Hampe
University Medical Center Hamburg-Eppendorf
Germany

BMC Med Educ 2017 Dec 11;17(1):246. Epub 2017 Dec 11.

Department of Biochemistry and Molecular Cell Biology, University Medical Center Hamburg-Eppendorf, Martinistraße 52, D-20246, Hamburg, Germany.

Background: The validity of selection tests is underestimated if it is determined by simply calculating the predictor-outcome correlation found in the admitted group. This correlation is usually attenuated by two factors: (1) the combination of selection variables which can compensate for each other and (2) range restriction in predictor and outcome due to the absence of outcome measures for rejected applicants.

Methods: Here we demonstrate the logic of these artifacts in a situation typical for student selection tests and compare four different methods for their correction: two formulas for the correction of direct and indirect range restriction, expectation maximization algorithm (EM) and multiple imputation by chained equations (MICE). First we show with simulated data how a realistic estimation of predictive validity could be achieved; second we apply the same methods to empirical data from one medical school.

Results: The results of the four methods are very similar except for the direct range restriction formula which underestimated validity.

Conclusion: For practical purposes Thorndike's case C formula is a relatively straightforward solution to the range restriction problem, provided distributional assumptions are met. With EM and MICE more precision is obtained when distributional requirements are not met, but access to a sophisticated statistical package such as R is needed. The use of true score correlation has its own problems and does not seem to provide a better correction than other methods.

Download full-text PDF

Source
http://dx.doi.org/10.1186/s12909-017-1070-5DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC5725878PMC

Still can't find the full text of the article?

We can help you send a request to the authors directly.
December 2017
8 Reads

Publication Analysis

Top Keywords

range restriction
20
selection tests
8
validity selection
8
indirect range
8
predictive validity
8
restriction
5
range
5
restriction formula
4
second apply
4
achieved second
4
validity achieved
4
data realistic
4
realistic estimation
4
estimation predictive
4
apply methods
4
direct range
4
medical schoolresults
4
schoolresults methods
4
data medical
4
empirical data
4

Similar Publications