Publications by authors named "Wes Bonifay"

9 Publications

  • Page 1 of 1

Development and validation of the Intervention Skills Profile-Skills: A brief measure of student social-emotional and academic enabling skills.

J Sch Psychol 2020 12 10;83:66-88. Epub 2020 Nov 10.

University of Wisconsin-Madison, United States of America.

The purpose of this study was to support the development and initial validation of the Intervention Selection Profile (ISP)-Skills, a brief 14-item teacher rating scale intended to inform the selection and delivery of instructional interventions at Tier 2. Teacher participants (n = 196) rated five students from their classroom across four measures (total student n = 877). These measures included the ISP-Skills and three criterion tools: Social Skills Improvement System (SSIS), Devereux Student Strengths Assessment (DESSA), and Academic Competence Evaluation Scales (ACES). Diagnostic classification modeling (DCM) suggested an expert-created Q-matrix, which specified relations between ISP-Skills items and hypothesized latent attributes, provided good fit to item data. DCM also indicated ISP-Skills items functioned as intended, with the magnitude of item ratings corresponding to the model-implied probability of attribute mastery. DCM was then used to generate skill profiles for each student, which included scores representing the probability of students mastering each of eight skills. Correlational analyses revealed large convergent relations between ISP-Skills probability scores and theoretically-aligned subscales from the criterion measures. Discriminant validity was not supported, as ISP-Skills scores were also highly related to all other criterion subscales. Receiver operating characteristic (ROC) curve analyses informed the selection of cut scores from each ISP-Skills scale. Review of classification accuracy statistics associated with these cut scores (e.g., sensitivity and specificity) suggested they reliably differentiated students with below average, average, and above average skills. Implications for practice and directions for future research are discussed, including those related to the examination of ISP-Skills treatment utility.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1016/j.jsp.2020.10.001DOI Listing
December 2020

The Milwaukee Youth Belongingness Scale (MYBS): Development and validation of the scale utilizing item response theory.

Sch Psychol 2019 May 17;34(3):296-306. Epub 2018 Dec 17.

Educational, School, and Counseling Psychology.

The examination of belonging in schools, connecting school belonging to a plethora of academic and psychosocial outcomes, has been well established in the literature. Researchers have measured school belonging most frequently with the Psychological Sense of School Membership, but its psychometric properties have been called into question by several researchers. Further, the scale measures 1 subset of belonging (i.e., school), leaving out powerful belonging connections in other areas of a student's life, namely peers and family. The current study examines the development and validation of the Milwaukee Youth Belongingness Scale. This process was examined by utilizing item response theory and a secondary analysis confirming the factor structure and the validation of the scale by comparing it to other constructs. The results confirm a 9-item scale that involves a total scale score and 3 factors (School, Peers, Family). Implications for mental health professionals and future research are discussed. (PsycINFO Database Record (c) 2019 APA, all rights reserved).
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1037/spq0000299DOI Listing
May 2019

Evidence for the interpretation of Social, Academic, and Emotional Behavior Risk Screener (SAEBRS) scores: An argument-based approach to screener validation.

J Sch Psychol 2018 06 19;68:129-141. Epub 2018 Mar 19.

University of Missouri, United States.

In accordance with an argument-based approach to validation, the purpose of the current study was to yield evidence relating to Social, Academic, and Emotional Behavior Risk Screener (SAEBRS) score interpretation. Bifactor item response theory analyses were performed to examine SAEBRS item functioning. Structural equation modeling (SEM) was used to simultaneously evaluate intra- and inter-scale relationships, expressed through (a) a measurement model specifying a bifactor structure to SAEBRS items, and (b) a structural model specifying convergent and discriminant relations with an outcome measure (i.e., Behavioral and Emotional Screening System [BESS]). Finally, hierarchical omega coefficients were calculated in evaluating the model-based internal reliability of each SAEBRS scale. IRT analyses supported the adequate fit of the bifactor model, indicating items adequately discriminated moderate and high-risk students. SEM results further supported the fit of the latent bifactor measurement model, yielding superior fit relative to alternative models (i.e., unidimensional and correlated factors). SEM analyses also indicated the latent SAEBRS-Total Behavior factor was a statistically significant predictor of all BESS subscales, the SAEBRS-Academic Behavior predicted BESS Adaptive Skills subscales, and the SAEBRS-Emotional Behavior predicted the BESS Internalizing Problems subscale. Hierarchical omega coefficients indicated the SAEBRS-Total Behavior factor was associated with adequate reliability. In contrast, after accounting for the total scale, each of the SAEBRS subscales was associated with somewhat limited reliability, suggesting variability in these scores is largely driven by the Total Behavior scale. Implications for practice and future research are discussed.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1016/j.jsp.2018.03.002DOI Listing
June 2018

Generalizability and Decision Studies of a Treatment Adherence Instrument.

Assessment 2020 03 2;27(2):321-333. Epub 2018 May 2.

Harvard University, Cambridge, MA, USA.

Observational measurement of treatment adherence has long been considered the gold standard. However, little is known about either the generalizability of the scores from extant observational instruments or the sampling needed. We conducted generalizability (G) and decision (D) studies on two samples of recordings from two randomized controlled trials testing cognitive-behavioral therapy for youth anxiety in two different contexts: research versus community. Two doctoral students independently coded 543 session recordings from 52 patients treated by 13 therapists. The initial G-study demonstrated that context accounted for a disproportionately large share of variance, so we conducted G- and D-studies for the two contexts separately. Results suggested that reliable cognitive-behavioral therapy adherence studies require at least 10 sessions per patient, assuming 12 patients per therapists and two coders-a challenging threshold even in well-funded research. Implications, including the importance of evaluating alternatives to observational measurement, are discussed.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1177/1073191118765365DOI Listing
March 2020

On the Complexity of Item Response Theory Models.

Authors:
Wes Bonifay Li Cai

Multivariate Behav Res 2017 Jul-Aug;52(4):465-484. Epub 2017 Apr 20.

b University of California , Los Angeles.

Complexity in item response theory (IRT) has traditionally been quantified by simply counting the number of freely estimated parameters in the model. However, complexity is also contingent upon the functional form of the model. We examined four popular IRT models-exploratory factor analytic, bifactor, DINA, and DINO-with different functional forms but the same number of free parameters. In comparison, a simpler (unidimensional 3PL) model was specified such that it had 1 more parameter than the previous models. All models were then evaluated according to the minimum description length principle. Specifically, each model was fit to 1,000 data sets that were randomly and uniformly sampled from the complete data space and then assessed using global and item-level fit and diagnostic measures. The findings revealed that the factor analytic and bifactor models possess a strong tendency to fit any possible data. The unidimensional 3PL model displayed minimal fitting propensity, despite the fact that it included an additional free parameter. The DINA and DINO models did not demonstrate a proclivity to fit any possible data, but they did fit well to distinct data patterns. Applied researchers and psychometricians should therefore consider functional form-and not goodness-of-fit alone-when selecting an IRT model.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1080/00273171.2017.1309262DOI Listing
April 2018

The recovery index: A novel approach to measuring recovery and predicting remission in major depressive disorder.

J Affect Disord 2017 Jan 15;208:369-374. Epub 2016 Oct 15.

UCLA Graduate School of Education and Information Studies, United States.

Background: Clinicians view "recovery" as the reduction in severity of symptoms over time, whereas patients view it as the restoration of premorbid functioning level and quality of life (QOL). The main purpose of this study is to incorporate patient-reported measures of functioning and QOL into the assessment of patient outcomes in MDD and to use this data to define recovery.

Method: Using the STAR*D study of patients diagnosed with MDD, this present analysis grades patients' MDD severity, functioning level, and QOL at exit from each level of the study, as well as at follow-up. Using Item Response Theory, we combined patient data from functioning and QOL measures (WSAS, Q-LES-Q) in order to form a single latent dimension named the Recovery Index.

Results: Recovery Index - a latent measure assessing impact of illness on functioning and QOL - is able to predict remission of MDD in patients who participated in the STAR*D study.

Conclusions: By incorporating functioning and quality of life, the Recovery index creates a new dimension towards measuring restoration of health, in order to move beyond basic symptom measurement.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1016/j.jad.2016.08.081DOI Listing
January 2017

Abstract: On the Complexity of IRT Models.

Authors:
Wes E Bonifay

Multivariate Behav Res 2015 ;50(1):128

a University of California , Los Angeles.

View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1080/00273171.2014.988988DOI Listing
March 2016

Scoring and modeling psychological measures in the presence of multidimensionality.

J Pers Assess 2013 2;95(2):129-40. Epub 2012 Oct 2.

Department of Psychology, University of California, Los Angeles, CA 90095, USA.

Confirmatory factor analytic studies of psychological measures showing item responses to be multidimensional do not provide sufficient guidance for applied work. Demonstrating that item response data are multifactorial in this way does not necessarily (a) mean that a total scale score is an inadequate indicator of the intended construct, (b) demand creating and scoring subscales, or (c) require specifying a multidimensional measurement model in research using structural equation modeling (SEM). To better inform these important decisions, more fine-grained psychometric analyses are necessary. We describe 3 established, but seldom used, psychometric approaches that address 4 distinct questions: (a) To what degree do total scale scores reflect reliable variation on a single construct? (b) Is the scoring and reporting of subscale scores justified? (c) If justified, how much reliable variance do subscale scores provide after controlling for a general factor? and (d) Can multidimensional item response data be represented by a unidimensional measurement model in SEM, or are multidimensional measurement models (e.g., second-order, bifactor) necessary to achieve unbiased structural coefficients? In the discussion, we provide guidance for applied researchers on how best to interpret the results from applying these methods and review their limitations.
View Article and Find Full Text PDF

Download full-text PDF

Source
http://dx.doi.org/10.1080/00223891.2012.725437DOI Listing
August 2013