An investigation of cross-cultural measurement invariance and item bias of pisa 2018 reading skills items

Abstract

The aim of this study is to investigate the cross-cultural measurement invariance of the reading skills items of the PISA 2018 at test and item level. Another aim of the study is to determine the item bias for items that do not show cross-cultural measurement invariance in line with expert opinions. A survey model was used in the study. The study sample consisted of a total of 3192 students who answered the questions of the Rapa Nui unit of reading skills test from Australia, China, and Turkey samples. Multi-Group Confirmatory Factor Analysis was conducted to examine the cross-cultural measurement invariance of the one-dimensional factor structure of the Rapa Nui unit. Item Response Theory Likelihood Ratio Test, Mantel Haenszel, Simultaneous Item Bias Test and Logistic Regression were used to investigate test items for Differential Item Functioning (DIF). As a result, the Rapa Nui unit of PISA 2018 showed only structural invariance according to countries in this study. The majority of the items showing differential item functioning were obtained from the comparison of Turkey and China. According to Australia-China and Australia-Turkey comparisons, five out of seven items were identified as showing DIF As a result of the bias study based on expert opinions, it was concluded that the items showed a bias in terms of the familiarity of a culture group with the content of the items, translation, the use of expressions in the item in different meanings, item format, and features measured by the item.