Application of the IRT residual DIF framework to ordinally scored polytomous items

초록

The study sought to extend the residual-based differential item functioning (RDIF) detection framework, originally developed for dichotomously scored items, to the analysis of ordinally scored polytomous items, ensuring fairness in testing. By leveraging the graded response model (GRM) for polytomously scored items, this research compared the RDIF framework against traditional polytomous DIF detection methods such as the Mantel, Generalized Mantel-Haenszel (GMH), and Item Response Theory-Likelihood Ratio Test (IRT-LRT) methods. Through an extensive simulation study manipulating factors like test length, sample size balance, percentage of DIF items, and impact of ability distributions, the study aimed to assess the performance of the RDIF framework in detecting DIF across various scenarios. Findings from the simulation revealed that the RDIF framework, especially the RDIF_RS statistic, demonstrated considerable effectiveness in identifying DIF with ordinally scored polytomous items, showing sufficient power and well-controlled Type I error rates. Although RDIF_RS exhibited slightly lower power than LRT and GMH in some conditions, The RDIF_RS has utility as an efficient and pragmatic method for evaluating DIF in polytomously scored items, considering how the implementation of RDIF framework was significantly swifter than that of LRT. Therefore, the simulation results advocate for the RDIF framework's practical utility in educational and psychological testing, proposing it as an efficient method for ensuring test fairness.