Search our Database of Scientific Publications and Authors

I’m looking for a

    Details and Download Full Text PDF:
    Correcting the predictive validity of a selection test for the effect of indirect range restriction.

    BMC Med Educ 2017 Dec 11;17(1):246. Epub 2017 Dec 11.
    Department of Biochemistry and Molecular Cell Biology, University Medical Center Hamburg-Eppendorf, Martinistraße 52, D-20246, Hamburg, Germany.
    Background: The validity of selection tests is underestimated if it is determined by simply calculating the predictor-outcome correlation found in the admitted group. This correlation is usually attenuated by two factors: (1) the combination of selection variables which can compensate for each other and (2) range restriction in predictor and outcome due to the absence of outcome measures for rejected applicants.

    Methods: Here we demonstrate the logic of these artifacts in a situation typical for student selection tests and compare four different methods for their correction: two formulas for the correction of direct and indirect range restriction, expectation maximization algorithm (EM) and multiple imputation by chained equations (MICE). First we show with simulated data how a realistic estimation of predictive validity could be achieved; second we apply the same methods to empirical data from one medical school.

    Results: The results of the four methods are very similar except for the direct range restriction formula which underestimated validity.

    Conclusion: For practical purposes Thorndike's case C formula is a relatively straightforward solution to the range restriction problem, provided distributional assumptions are met. With EM and MICE more precision is obtained when distributional requirements are not met, but access to a sophisticated statistical package such as R is needed. The use of true score correlation has its own problems and does not seem to provide a better correction than other methods.
    PDF Download - Full Text Link
    ( Please be advised that this article is hosted on an external website not affiliated with
    Source Status ListingPossible

    Similar Publications

    A Missing Data Approach to Correct for Direct and Indirect Range Restrictions with a Dichotomous Criterion: A Simulation Study.
    PLoS One 2016 28;11(3):e0152330. Epub 2016 Mar 28.
    Department of Applied Psychology: Work, Education, Economy, Faculty of Psychology, University of Vienna, Vienna, Austria.
    A recurring methodological problem in the evaluation of the predictive validity of selection methods is that the values of the criterion variable are available for selected applicants only. This so-called range restriction problem causes biased population estimates. Correction methods for direct and indirect range restriction scenarios have widely studied for continuous criterion variables but not for dichotomous ones. Read More
    Construct-level predictive validity of educational attainment and intellectual aptitude tests in medical student selection: meta-regression of six UK longitudinal studies.
    BMC Med 2013 Nov 14;11:243. Epub 2013 Nov 14.
    UCL Medical School, University College London, Gower Street, London WC1E 6BT, UK.
    Background: Measures used for medical student selection should predict future performance during training. A problem for any selection study is that predictor-outcome correlations are known only in those who have been selected, whereas selectors need to know how measures would predict in the entire pool of applicants. That problem of interpretation can be solved by calculating construct-level predictive validity, an estimate of true predictor-outcome correlation across the range of applicant abilities. Read More
    Testing the generalizability of indirect range restriction corrections.
    J Appl Psychol 2014 Jul 24;99(4):587-98. Epub 2014 Mar 24.
    Department of Psychology, University of Minnesota.
    Range restriction is a common problem in personnel selection and other contexts in applied psychology. For many years researchers have used corrections that assume range restriction was direct, even when it was known that range restriction was indirect. Hunter, Schmidt, and Le (2006) proposed a new correction for cases of indirect range restriction that greatly increases its potential usefulness due to its reduced information requirements compared to alternatives. Read More
    The UKCAT-12 study: educational attainment, aptitude test performance, demographic and socio-economic contextual factors as predictors of first year outcome in a cross-sectional collaborative study of 12 UK medical schools.
    BMC Med 2013 Nov 14;11:244. Epub 2013 Nov 14.
    UCL Medical School, University College London, Gower Street, London WC1E 6BT, UK.
    Background: Most UK medical schools use aptitude tests during student selection, but large-scale studies of predictive validity are rare. This study assesses the United Kingdom Clinical Aptitude Test (UKCAT), and its four sub-scales, along with measures of educational attainment, individual and contextual socio-economic background factors, as predictors of performance in the first year of medical school training.

    Methods: A prospective study of 4,811 students in 12 UK medical schools taking the UKCAT from 2006 to 2008 as a part of the medical school application, for whom first year medical school examination results were available in 2008 to 2010. Read More