Correcting the predictive validity of a selection test for the effect of indirect range restriction.

Authors:
Stefan Zimmermann
Stefan Zimmermann
University of Heidelberg
Germany
Dietrich Klusmann
Dietrich Klusmann
Universit├Ątsklinikum Hamburg-Eppendorf
Germany
Wolfgang Hampe
Wolfgang Hampe
University Medical Center Hamburg-Eppendorf
Germany

BMC Med Educ 2017 Dec 11;17(1):246. Epub 2017 Dec 11.

Department of Biochemistry and Molecular Cell Biology, University Medical Center Hamburg-Eppendorf, Martinistra├če 52, D-20246, Hamburg, Germany.

Background: The validity of selection tests is underestimated if it is determined by simply calculating the predictor-outcome correlation found in the admitted group. This correlation is usually attenuated by two factors: (1) the combination of selection variables which can compensate for each other and (2) range restriction in predictor and outcome due to the absence of outcome measures for rejected applicants.

Methods: Here we demonstrate the logic of these artifacts in a situation typical for student selection tests and compare four different methods for their correction: two formulas for the correction of direct and indirect range restriction, expectation maximization algorithm (EM) and multiple imputation by chained equations (MICE). First we show with simulated data how a realistic estimation of predictive validity could be achieved; second we apply the same methods to empirical data from one medical school.

Results: The results of the four methods are very similar except for the direct range restriction formula which underestimated validity.

Conclusion: For practical purposes Thorndike's case C formula is a relatively straightforward solution to the range restriction problem, provided distributional assumptions are met. With EM and MICE more precision is obtained when distributional requirements are not met, but access to a sophisticated statistical package such as R is needed. The use of true score correlation has its own problems and does not seem to provide a better correction than other methods.

Download full-text PDF

Source
http://dx.doi.org/10.1186/s12909-017-1070-5DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC5725878PMC

Still can't find the full text of the article?

We can help you send a request to the authors directly.
December 2017
13 Reads

Publication Analysis

Top Keywords

range restriction
20
selection tests
8
validity selection
8
indirect range
8
predictive validity
8
restriction
5
range
5
restriction formula
4
second apply
4
achieved second
4
validity achieved
4
data realistic
4
realistic estimation
4
estimation predictive
4
apply methods
4
direct range
4
medical schoolresults
4
schoolresults methods
4
data medical
4
empirical data
4

References

(Supplied by CrossRef)
Article in J Appl Psych
PR Sackett et al.
J Appl Psych 2000

WN Venables et al.
2002
Article in BMC Med
I McManus et al.
BMC Med 2013
Article in Am Stat
L Friedman et al.
Am Stat 2005
Article in Educ Psych Meas.
AJ Conger et al.
Educ Psych Meas. 1974

J Cohen et al.
2003
Article in Educ Psych Meas
WF Velicer et al.
Educ Psych Meas 1968
Article in Gen Pract
F Patterson et al.
Gen Pract 2013
Article in BMJ Open
RK MacKenzie et al.
BMJ Open 2016
Article in Appl Psych Meas.
AL Gross et al.
Appl Psych Meas. 1983

Similar Publications